SYSTEM FOR PROCESSING A DIGITAL IMAGE USING TWO OR MORE DEFINED REGIONS
System for processing a digital image using multiple regions where each of the multiple regions is defined using a portion of the digital image. In addition, a top margin, a bottom margin, a left margin, and a right margin are defined to include margin pixels for each of the multiple regions of the digital image. Margin pixels are defined as neighboring pixels for a region of the digital image and may contain pixels that are part of the digital image, part of the region of the digital image, and/or newly generated pixels. Various techniques such as on the fly generation or using a predetermined process for video data information generation, such as replication or using a fixed color, may be used to generate the margin pixels. Each of the multiple regions is processed with its margin pixels to create a new quadrilateral digital image that is completely processed and/or scaled. The appropriate portion of the processed quadrilateral digital image is displayed using a display region of an electronic display panel having multiple display regions. The concatenation of the displayed portions of the processed quadrilateral digital images produces the original digital image using the electronic display panel.
Latest Marseille Networks, Inc. Patents:
This application is a continuation of U.S. patent application Ser. No. 12/800,266, filed May 12, 2010, entitled “Apparatus For Partitioning and Processing A Digital Image Using Two Or More Defined Regions”, which is a continuation-in-part of U.S. patent application Ser. No. 12/655,769, filed Jan. 6, 2010, entitled “Method For Partitioning A Digital Image Using Two Or More Defined Regions”, both applications are incorporated by reference herein in their entirety.
TECHNICAL FIELDThe subject of this application generally relates to the field of digital image processing and more particularly to digital image scaling from an original image size to a target image size.
BACKGROUNDTraditional cameras are used to take photographs by capturing light onto photographic films. Digital cameras use electronic devices such as Charge Coupled Devices (CCD) to capture light, of an original image, and produce a digital representation of an image. The digital representation can be further manipulated using different compression or transmission techniques or standards such as Moving Picture Experts Group (MPEG). Furthermore, the digital representation of the image can be stored in various digital formats in accordance with the intended memory storage mediums e.g. Hard disk, DVD, CD-Rom, etc. . . . such that the reproduction of the original image may be achieved using a variety of means or mediums using the stored digital representation of the original image. For examples, electronic display devices can display the image on their screen.
The digital representation of the image can greatly vary in quality depending on the sampling of the original image. Each sample of the original image represents a small portion of the overall image. Therefore, more samples are required in order to have a better or more accurate representation of the original image. A pixel represents one sample of the original image. Normally, an image is sampled using a two-dimensional grid having a width, X, and a height, Y, that are specified in unit of pixel, where the digital image resolution corresponds to X time Y, and each pixel corresponds to the smallest single component of the original image. For example, a first camera with a resolution of 640×480 would have 640 pixels in the horizontal direction and 480 pixels in the vertical direction. The digital image resolution, total number of pixels, is 307,200 pixels. Higher resolution requires more pixels to be generated when capturing an image, and the closer the digital image produced is to the original image. Hence, a second digital camera with a resolution of 1280×960 would have a total number of pixels of 1,228,800 pixels or four times the resolution of the first camera.
Each pixel of a digital image corresponds to data information that is expressed as a number of bits that is used to describe each pixel (or sample) of the original image. This data information is normally expressed as number of bits per pixel (bpp). A broader range of distinct colors can be represented with a higher number of bits per pixel. Nowadays, There are many different formats that are in use to capture and/or display color information, e.g. the RGB. For example, a 24-bit color model uses 8 bits to represent red, 8 bits to represent blue and 8 bits to represent green. Under this model, each of these three colors possesses a 28 or 256 levels. Therefore, they can be combined (256×256×256) to give a possible 16,777,216 colors.
A video camera captures a scene for a specific duration of time, and produces many sequential digital images. Normally, each digital image is referred to as a frame, having a frame size defined as number of horizontal pixels×number of vertical pixels. A frame rate is also specified that represents the number of frames being captured per second. In addition, a scanning system is identified as progressive or interlaced to indicate how the video frames are generated and thus how they should be processed and displayed so that the original scene is faithfully reproduced when these digital images are played back in sequence, e.g. using an electronic display panel or a digital television set.
In order to reproduce the original scene timing, each digital image, or frame within the scene, must be reproduced and displayed in a given amount of time. Hence, the time required to process and display one pixel is limited and finite. Electronic display devices resolution is specified, in a similar way as explained above for a digital camera, as having X by Y pixels. Again, the higher the resolution of the electronic display device is, the better the image that is being reproduced. As the electronic display panel technology advances to an ever-higher resolution, a bigger challenge to the device electronics is to be able to process data information for each pixel within an ever-smaller amount of time.
The processing demands on electronic circuits for High-Definition television (HD TV), e.g. 1,920 pixels wide and 1,080 pixels high, is much greater than a Standard-Definition television (SD TV), e.g. 720 pixels wide and 480 pixels high. The next generation of digital TVs and projectors, recently developed, will be able to display four times the high definition resolution of current HD TV sets. This Quad-HD set is capable of displaying 3,840 pixels wide and 2,160 pixels high. This presents a big challenge to the processing circuitry, where each pixel must be processed and faithfully reproduced regardless of the input image resolution having a Quad-HD content from standardized interfaces such as HDMI 1.4 or DisplayPort 1.2a and/or other lower resolution such as 1,920×1,080 or lower.
The need arises to provide an electronic system capable of faithfully processing digital images with various resolutions and display them using a Quad-HD resolution display or higher resolution type of electronic display panels. Marseille Networks' 4×HD™ video technology delivers the ability to process digital images to be displayed in 3840×2160 resolution, while selectively removing artifacts and preserving stunning image details. Furthermore, Marseille Networks is the first to introduce Quad-HD solution to home theater systems. Marseille Networks' 4×HD™ video technology provide an efficient system with ample flexibility and processing power for blending and or scaling various types of video image streams, including High-Definition streams, to be displayed over Quad-HD display panel.
The present disclosure provides, amongst other things, techniques to minimize or substantially eliminate artifacts when scaling low resolution video or digital images to higher resolution digital images in order to be displayed using electronic display panels with two or more defined regions. In one respect, a digital video comprises many sequential digital images that capture a certain scene for a certain period of time. A digital image is split into multiple regions and each region is then selectively processed and/or scaled so that it can be displayed using a correspondent region of the display. Each of the multiple regions is defined using a portion of the digital image and is specified using a width and a height, in unit of pixel. In addition, neighboring pixels for each of the multiple regions are defined as margins, e.g. Top margin, Right margin, Bottom margin, and Left margin that will be described in details in this disclosure, and are specified to comprise pixels that are part of the digital image and/or newly generated pixels using various techniques such as on the fly or a predetermined data information. The combination of each of the multiple regions and all of its margins are then processed or scaled to produce a processed portion of the digital image. Furthermore, the concatenation of all of the processed portions of the digital image are then displayed, using electronic display panels with multiple defined regions, and thus faithfully reproduce the original digital image.
The disclosure and the various features and advantageous details are explained more fully with reference to the nonlimiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known processing techniques, components, and equipment are omitted so as not to unnecessarily obscure the invention in detail. It should be understood, however, that the detailed description and the specific examples, while indicating embodiments of the invention, are given by way of illustration only and not by way of limitation. Various substitutions, modifications, additions, and/or rearrangements within the spirit and/or scope of the underlying inventive concept will become apparent to those of ordinary skill in the art from this disclosure. Other features and associated advantages will become apparent with reference to the following detailed description of specific embodiments in connection with the accompanying drawings.
The term “coupled” is defined as connected, although not necessarily directly, and not necessarily mechanically. The terms “a” and “an” are defined as one or more unless this disclosure explicitly requires otherwise. The terms “comprise” (and any form of comprise, such as “comprises” and “comprising”), “have” (and any form of have, such as “has” and “having”), “include” (and any form of include, such as “includes” and “including”) and “contain” (and any form of contain, such as “contains” and “containing”) are open-ended linking verbs. As a result, a method or device that “comprises,” “has,” “includes” or “contains” one or more steps or elements possesses those one or more steps or elements, but is not limited to possessing only those one or more elements. Likewise, a step of a method or an element of a device that “comprises,” “has,” “includes” or “contains” one or more features possesses those one or more features, but is not limited to possessing only those one or more features. Furthermore, a device or structure that is configured in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
In general, various processing techniques can be used to process or scale digital images from one specified resolution to another specified resolution. For example, newly available Quad-HD electronic display panels, or can be simply referred to as monitors, can display images using a resolution of 3840×2160 pixels, or 3840 pixels in width and 2160 pixels in height, for a total of 8,294,400 pixels. A high definition digital image may be specified using 1920 pixels in width and 1080 pixels in height, or in other words a resolution of 1,920×1,080 for a total of 2,073,600 pixels. As you can see The Quad-HD monitor is capable of displaying four times the number of pixels that define the digital image using four regions each of which is capable of displaying a full 1,920×1,080 pixels. Normally a low resolution digital image is first scaled to 3840×2160 resolution and then split into four regions, each of which is then displayed using a corresponding region of the monitor. The higher the resolution of an available Quad-monitor is, the higher the processing requirement is to display a given digital image at the full monitor resolution. This disclosure describes a suitable solution to solve the ever-increasing challenge of scaling a given digital image in order to be faithfully reproduced and fully displayed using monitors with multiple defined regions, e.g. Quad-HD monitor.
An exemplary model depicting a digital image 100 using pixels arranged as a two-dimensional grid is shown in
Now referring to
Referring to
Referring to
In one embodiment, the second region of the quadrilateral digital image 305 is determined using a Top margin 335, a Right margin 345, a Bottom margin 355, and a Left margin 365 using the same number of pixels. For example, each of the Top margin 335, the Right margin 345, the Bottom margin 355, and the Left margin 365 comprises at least one pixel. An exemplary margin of two pixels is used for all Top, Right, Bottom, and Left margins, as shown in
Q1 Width 314=(Left Margin 365+D1 Width 312+Right Margin 345) Eq. 1
Q1 Height 324=(Top Margin 335+D1 Height 322+Bottom Margin 355) Eq. 2
Accordingly, for the exemplary drawing shown in
In accordance with one embodiment, the data information for each pixel located in any of the Top margin 335, the Right margin 345, the Bottom margin 355, and the Left margin 365 is generated using existing data information from any one pixel of the quadrangle digital image D1 380, or using predetermined color data information, or any combination thereof. The predetermined color data information may corresponds to any color including black color. In accordance with another embodiment, a Top Wing 337 is defined to comprise at least one row of pixels that are located directly above the Top Edge 332. For example, the Top Wing 337 comprises two pixels, identified as Top Wing e-pixel 338, that are immediately above the correspondent pixel, Top Edge e-pixel 333, in the vertical direction, as shown in
In accordance with a preferred embodiment, a Left Wing 367 comprises at least one column of pixels that is located immediately to the left of the Left Edge 362 of the quadrangle digital image D1 380. The data information for the Left Wing 367 pixels is generated using data information of the corresponding pixels that are identified as the leftmost pixels of the Top Wing 337, Top Wing e-pixel 338 as shown the in
In another embodiment, changing the order of the data information that is generated first, i.e. Right Wing 347 & the Left Wing 367, and then data information for the Top Wing 337 & the Bottom Wing 357 would result in a change of the number of pixels being processed in each of the steps identified above for each one of the Right Wing 347, the Left Wing 367, the Top Wing 337, and the Bottom Wing 357 but does not change the underlying inventive concepts as described above and this flexibility would be appreciated by those skilled in the art. As will be described and shown in the following paragraphs, the definition of the quadrilateral digital image 305 will be used in the processing of an arbitrary digital image, which is split into multiple quadrangle regions, D1, D2, D3, . . . , Dz. The margin pixels are then either determined or created, as described above, for each of the multiple quadrangle regions, Dz. Sometimes these margin pixels are part of the arbitrary digital image and sometimes the margin pixels are generated based on any one, or any combination, of methods as described above. A new quadrilateral digital image, Qz, is then created from the concatenation of each of the multiple quadrangle regions, Dz, and its own margin pixels.
Now referring to
Therefore, data information is generated for certain margin pixels that are determined to be beyond the Topmost edge 330 and the Leftmost edge 360, see
Now referring to
An electronic display panel, may be referred to as monitor, are used in an ever increasing number of applications that generates very high resolution digital images or demands the ability to display multiple high resolution images. One such exemplary monitor is shown in
An exemplary process of displaying an Image 510 using a Scaled image 515 and a quad display Monitor 595 will be described as shown
Referring to
Now referring to
A Digital image 710 having a topmost edge, a bottommost edge, a leftmost edge, and a rightmost edge is split into M rows and N columns, and thus defining an M×N array of quadrangle digital images, each of which is identified as D(m,n), as shown in
Margin pixels are determined for an exemplary D(1,2), D(m,n) and D(M,N) as shown in
Q(m,n)width=Left margin+D(m,n)width+Right margin, Eq. 3
Q(m,n)height=Top margin+D(m,n)height+Bottom margin, Eq. 4
Each of the quadrangle digital images D(m,n) is centered within the quadrilateral digital image Q(m,n) using the Top margin 751, the Bottom margin 753, the Left margin 754, and the Right margin 752. In addition, at least a first pixel of the quadrilateral digital image Q(m,n) is determined to be located beyond any one of the topmost edge, the bottommost edge, the leftmost edge, and the rightmost edge of the Digital image 700 if any one of a D(m,n) top edge, a D(m,n) bottom edge, a D(m,n) left edge, and a D(m,n) right edge of a quadrangle digital image D(m,n), is collinear with any one of the topmost edge, the bottommost edge, the leftmost edge, and the rightmost edge of the Digital image 700. Appropriate data information is generated for the first pixel based on at least one of the following steps: (i) using data information of at least one pixel that is located on any one of the D(m,n) top edge, the D(m,n) bottom edge, the D(m,n) left edge, and the D(m,n) right edge of the correspondent quadrangle digital image D(m,n), (ii) using data information of at least one pixel that is located on any one of the topmost edge, the bottommost edge, the leftmost edge, and the rightmost edge of the Digital image 700, and (iii) using data information that corresponds to any color, including black.
In a similar fashion, data information for all margin pixels is determined first by checking if the margin pixels are within the Digital image 700 or are beyond any one of the topmost edge, the bottommost edge, the leftmost edge, and the rightmost edge of the Digital image 700. Various method of generating and/or creating data information for margin pixels have been described in details in this disclosure and used here for every quadrilateral digital image Q(m,n). In accordance with another embodiment, it should be noted that the general solution described above for splitting any Digital image 700 into an array of M×N array of quadrangle digital images is not limited to two-dimensional arrangement of all elements of the array. For example, multi-dimensional arrangement of (M×N×Z) array of quadrangle digital images would benefit greatly and the inventive concepts here are easily applied to each element of the multi-dimensional arrangement of array that corresponds to a quadrangle digital image.
A block diagram of a Digital Image Processing System 800 is shown in
The Memory Data Path Controller 830 handles as many memory access requests as needed for all the output display modules ODM—1 through ODM_k, which are part of the Output Module 840, so that the video data information of the Digital Image 900 is processed as desired. The access efficiency to and from Memory 850 has a large impact on the scalability of the Digital Image Processing System 800, and the overall video data information processing throughput. Various modes or types of memory access are used and the Memory Data Path Controller 830 provides the ability to handle different types of back-to-back memory access requests. The design and performance of the Second Memory Data Path 832 and the First Memory Data Path 831 contribute as well to the overall performance and efficiency desired. The component, modules and functional description of the Digital Image Processing System 800, as shown in
The Input Module 810 comprises three main components, an Input Capture 810—10 coupled to the Digital Input Data Path 811 and a Video Formatter 810—20, and a Memory Interface 810—30 coupled to the First Memory Data Path 831 and the Video Formatter 810—20, as shown in the block diagram of
1. RGB 4:4:4, 24 bits per pixel
2. YUV 4:4:4, 24 bits per pixel
3. YUV 4:2:2, 16 bits per sample, YU, YV, YU etc.
4. YUV 4:2:2, 8 bits per sample, U, Y, V, Y, U etc.
In addition, the video format standard specifies display control information such as Vertical and Horizontal synchronization, Vsync and Hsync, information that corresponds to and provides information regarding how the pixels of the input digital image are arranged in accordance with the resolution and the video format standard of the input digital image. A pixel data enable signal, which may be triggered from the Vsync signal, may be added to indicate the presence of valid lines (or rows) of pixels to be captured or displayed. Alternatively, a pixel data enable signal, which may be triggered from the Hsync signal, may be used to indicate the presence of valid pixels, within a line (or rows) of pixels, to be captured or displayed. Thus, the ability to control the blanking, or the display, of a portion of the input digital image is enhanced by using appropriate data enable signals. Furthermore, the Digital Input Data Path 811 comprises at least one channel to transfer video data information signals to be received by the Input Capture 810—10 from a digital image source. The channel may comprise one or multiple conductors to carry the video data information signal in accordance with various desired signaling schemes, e.g. Low Voltage Differential Signaling or LVDS. In certain application, some digital image sources may require multiple channels to carry the video data information of the digital image; this is especially true whenever the digital image resolution is large enough such that the available channel bandwidth is not large enough to enable the transfer of the required video data information. In a preferred embodiment, the Digital Input Data Path 811 comprises four input channels, where each channel comprises five differential pairs of conductor and can operate at a variable frequency rate in accordance with the video format standard of the input video data information stream, e.g. 24-bit RGB 4:4:4 operating at a frequency of 74.25 MHz. In accordance with another embodiment, the output of multiple digital image sources may be coupled to the Digital Input Data Path 811, where at least one channel may be used to transfer their video data information to one or multiple instances of the Input Capture 810—10 module, as will be described and discussed later on in this disclosure.
The Input Module 810 further comprises logic circuitry to receive or generate a first clock that is associated with the video data information. The Input Capture 810—10, Video Formatter 810—20 and Memory Interface 810—30 may use this first clock for their operation. Multiples clocks may be generated from, or in addition to, the first clock as may be required. For example, the first clock may be inputted to a Phase Locked-Loop (PLL) or other types of circuitry to maintain and distribute clocks to other components of the Digital Image Processing System 800. An output video display clock may also be derived from or linked to the first clock, thus providing the ability for the output display to be locked to the input video data information stream. As it has been mentioned earlier, various blocks or components may not be shown or discussed since their details is not necessary for a person of ordinary skills in the art to understand and practice the invention. Furthermore, the Digital Image Processing System 800 may comprise a central processing unit CPU and its associated components, e.g. software, firmware and hardware, where some of the Digital Image Processing System 800 operations may be executed, managed, defined, substituted for one or more modules, configured or programmed to produce the desired system functions.
In accordance with one embodiment, the Video Formatter 810—20 receives a first digital image that is captured by the Input Capture 810—10 using a first video format, e.g. RGB 4:4:4, having twenty-four bits for every pixel of the digital image. The Video Formatter 810—20 generates at least a second and third digital image, based on a second video format e.g. YUV 4:2:2, using the first digital image received. The second digital image comprises the luminance video data information (Y) of the digital image, while the third digital image comprises the chrominance video data information (UV) of the first digital image. The Memory Interface 810—30 generates at least one first memory write command to transfer the video data information of the second digital image to the Memory Data Path Controller 830 via the First Memory Data Path 831. The total number of memory write command required to transfer the video data information of the second digital image depends on the architecture of Digital Image Processing System 800. Various techniques may be used where one or multiple memory write commands may be used to effect the complete transfer of the video data information from the Input Module 810 to the Memory 850. An exemplary transfer of video data information to the Memory 850 is described in the following paragraphs.
The Memory Data Path Controller 830 receives the first memory write command via the First Memory Data Path 831 and generates at least one first memory device write command to effect a transfer of the video data information of the second digital image to the Memory 850 via the Second Memory Data Path 832. The first memory write command comprises a first base index that is used by the Memory Data Path Controller 830 to generate a first base address that indicates a starting address of a first region within the Memory 850, where the video data information of the second digital image is to be stored. The Memory Data Path Controller 830 embeds the first base address within the first memory device write command. Furthermore, the Memory 850 comprises control circuitry to receive the first memory device write command, and further comprises receive circuitry to receive the second video data information via the Second Memory Data Path 832. The Memory 850 writes the second video data information into the first region of the Memory 850 starting from the memory location as indicated by the first base address. Additional data information, which may be embedded in the memory device write or read commands, may be used to further specify an offset address for the starting memory location or how many memory locations are to be used for storing the video data information, as will be described the paragraphs below.
The Memory Interface 810—30 generates at least one second memory write command to transfer the video data information of the third digital image to the Memory Data Path Controller 830 via the First Memory Data Path 831. The Memory Data Path Controller 830 receives the second memory write command via the First Memory Data Path 831 and generates at least one second memory device write command to effect a transfer of the video data information of the third digital image to the Memory 850 via the Second Memory Data Path 832. The second memory write command comprises a second base index that is used by the Memory Data Path Controller 830 to generate a second base address that indicates a starting address of a second region within the Memory 850, where the video data information of the third digital image is to be stored. The Memory Data Path Controller 830 embeds the second base address within the second memory device write command. Furthermore, the Memory 850 comprises control circuitry to receive the second memory device write command, and further comprises receive circuitry to receive the third video data information via the Second Memory Data Path 832. The Memory 850 writes the third video data information into the second region of the Memory 850 starting from the memory location as indicated by the second base address.
Now referring to
The Register and Control Module 820 provides the necessary access and operational settings to various programmable components, e.g. registers, within the Digital Image Processing System 800 via the First Control Data Path 821. In addition, the Second Control Data Path 822 provides the necessary interface, e.g. I2C, for an external device to access and set various programmable registers of various components and modules of the Digital Image Processing System 800 via the First Control Data Path 821. The Register and Control Module 820 performs all bus protocols to read and write internal registers, with both round-robin arbitration and a full acknowledge handshake to enable cross-clock-domain operation. Furthermore, the Digital Image Processing System 800 may comprise a Read Only Memory (ROM) that contains the default configuration that the Register and Control Module 820 uses to program various modules on power-up. Hence, relieving an external device from using the Second Control Data Path 822 to program the Digital Image Processing System 800 registers to a default power-up state. The Register and Control Module 820 can service multiple clients within the Digital Image Processing System 800 via the First Control Data Path 821 and at least one external client via the Second Control Data Path 822. In the following paragraphs, an exemplary write and read transactions through the Register and Control Module 820 are briefly described.
A source module set a request for a write command for a specific register of a specific module. The address of the specific register and the address of the specific module are embedded within the write command. The source module can be any one of the Digital Image Processing System 800 internal modules, where the write command is sent via the First Control Data Path 821, or an external module, where the write command is sent via the Second Control Data Path 822. The destination module can be any one of the Digital Image Processing System 800 internal modules. The Register and Control Module 820 detects the write command and in turn places the write address for that request on a register address bus portion of the First Control Data Path 821, and places the write data on a register write data bus portion of the First Control Data Path 821. The Register and Control Module 820 then activates the decoded write strobe for the destination module. Clock synchronization may be implemented to insure the destination module will receive proper write strobe, e.g. dual-rank synchronization of the write strobe. In response to the write strobe, the destination module samples the data, registers address, and writes the data to the appropriate internal register location within the destination module as indicated by the register address. At that time, the destination module may send an acknowledge signal to the Register and Control Module 820. The destination module may hold the acknowledge signal true until it detects the release of the write strobe from the Register and Control Module 820. The Register and Control Module 820 may further dual-rank synchronizes the returning acknowledge signal before using it to decide to release the write strobe. Once the write strobe is released, the Register and Control Module 820 is able to perform another read or write transaction. In an alternate embodiment, the write strobe may be used to designate a write cycle to multiple registers at the same time. The write command specifies a broadcast type of command indicating a range of registers to be written at the same time with the same data. In yet another embodiment, the write strobe may be used to designate a write cycle to multiple registers that are located in different modules at the same time. The write command specifies a broadcast type of command indicating the address of one or multiple registers to be written. All the destination modules that receives the broadcast type write command, initiate a write cycle to their register as identified by the register address within the command cycle. This can produce an efficient and fast way to reconfigure Digital Image Processing System 800, e.g. when image size changes or when a change in video format standard is detected or required. An exemplary register read request is described in the next paragraph.
A source module set a request for a read command of a specific register within a specific module. The address of the specific register and the address of the specific module are embedded within the read command. The source module can be any one of the Digital Image Processing System 800 internal modules, where the read command is sent via the First Control Data Path 821, or an external module, where the read command is sent via the Second Control Data Path 822. The destination module can be any one of the Digital Image Processing System 800 internal modules. The Register and Control Module 820 detects the read command and in turn places the read address for that request on the register address bus portion of the First Control Data Path 821. The Register and Control Module 820 then activates the decoded read strobe for the destination module. Read strobe synchronization is achieved in a similar fashion to the write strobe synchronization, as described above, e.g. dual-rank synchronization of the read strobe. The destination module places the read data on a read data bus portion of the First Control Data Path 821. At that time, the destination module may asserts an acknowledge signal and send it to the Register and Control Module 820. The destination module will hold the acknowledge signal true until it detects the read strobe from the Register and Control Module 820 going false. The Register and Control Module 820 may dual-rank synchronizes the returning acknowledge signal before making the decision to release the read strobe. Once the read strobe is released, the Register and Control Module 820 is able to perform another read or write transaction. It is important to note that the First and Second Control Data Path 821 and 822 may use various types of handshake control. A dedicated portion of the First and Second Data Path 821 and 822 may be used exclusively for any of the address, data, and control signals or may be used in a multiplexed fashion depending on the desired design and system performance criteria. An alternate embodiment of Register and Control Module 820 includes a dedicated point-to-point read data bus and a dedicated party-line write data bus that are implemented between the Register and Control Module 820 and the Digital Image Processing System 800 modules via the First and Second Control Data Path 821 and 822. The following paragraphs describe the operation of the Output Module 840.
An exemplary electronic display panel may comprise many regions and each of which is independently driven. Therefore as many output display modules (ODM) as display regions are required, where each ODM retrieves, processes, and outputs onto the Output Display Data Path 846 the processed video data information for its designated display region. In accordance with one embodiment, the architecture of the Digital Image Processing System 800 is developed with the ability to drive a large display panel that comprises sixteen display regions. In order to provide such a flexible system architecture to be used with various types of electronic display panels, a unique and programmable design features are implemented within each ODM. One of the ODM design features is that each ODM comprises identical programmable circuitry, where each ODM can be programmed, e.g. enumerated using sequential numbers 0, 1, 2 . . . , such that only one of the ODMs will behave as a Master ODM and all of the others will behave like Slave ODMs. One of the functions of the Master ODM is to control and insure that the starting point of the Vertical and Horizontal sync signals is the same for all of the ODMs. In addition, the master ODM further controls and intervenes in the multiplexing of the data being read from specific registers within each of the Slave ODMs through the First Control Data Path 821. Any of the ODMs comprises the circuitry to operate as a Master ODM if it is programmed as such. Therefore, the ability to scale the Digital Image Processing System 800 to drive large number of display regions with as many numbers of ODMs is greatly enhanced by the simple functional ability to enumerate each ODM and program its functionality. Some additional advantages are gained when scaling the Digital Image Processing System 800 such as: (i) a reduction of the amount of time required to test the Output Module 840 and to verify the functionality of each ODM, and (ii) correct routing of each ODM within the overall Digital Image Processing System 800.
The Output Module 840, as shown in
A horizontal or vertical scaling ratio of one indicates that the output resolution in the specified dimension, i.e. horizontal or vertical, is the same as the input resolution and no scaling takes place. However, each of the ODM—1 841, ODM—2 842, ODM—3 843, ODM—4 844, and ODM_k 845 comprises programmable circuitry capable of processing and scaling the retrieved digital image using an arbitrary horizontal scaling ratio and an arbitrary vertical scaling ratio, that do not necessarily need to be equal. This is especially beneficial in processing digital images using various video format standards that may specify different vertical and horizontal resolutions. Thus, different vertical and horizontal scaling ratios are used in order to properly process and display the digital image using the specified display region resolution. This is illustrated in the following example. A first digital image with a resolution of 720×480 is to be displayed using the display region, Monitor 660 as shown in
Various types of electronic display monitors with multiple defined regions may be coupled to the Output Display Data Path 846 to receive and to display the processed video data information of the digital images processed by each of the output display modules. Furthermore, an electronic display monitor may comprise an arbitrary number of display regions as may be specified by M rows and N columns e.g. see
In accordance with one embodiment, the Output Display Data Path 846 comprises four independent channels. Each channel is used to couple one of the output display modules, namely ODM—1 841, ODM—2 842, ODM—3 843, and ODM—4 844, to one of the regions of the electronic display panel Monitor 695, namely Monitor 660, Monitor 670, Monitor 680, and Monitor 690. Each of the ODM—1 841, ODM—2 842, ODM—3 843, and ODM—4 844 comprises identical circuitry and independently retrieves and processes a designated portion of the Image 610, namely the Split 620, Split 630, Split 640, and Split 650 respectively, as described above and in
The Input Module 810, in accordance with one embodiment, is shown in
The Memory Data Path Controller 830, the First Memory Data Path 831, and the Second Memory Data Path 832 design requirement is to efficiently transfer large amounts of data between the various modules of Digital Image Processing System 800, e.g. between the Input Module 810 and the Output Module 840, and the Memory 850. Control and video data information are pipelined together and thus pipeline stages can be added to achieve timing closure especially when the Digital Image Processing System 800 is being manufactured as a System-on-Chip or SoC. In this case, large amount of data must be transferred relatively long distances throughout the SoC. Arbitrary numbers of pipeline stages between the Memory Data Path Controller 830 and Digital Image Processing System 800 modules can be added without affecting the performance. The Memory Data Path Controller 830 is designed to accommodate and respond to different modes of operation for each of the internal modules of Digital Image Processing System 800, and thus enabling efficient and high performance access to and from the Memory 850 using both the First and Second Memory Data Path 831 and 832 respectively. Each module can store or retrieve data from the Memory 850 in the most efficient format for its data processing or type usage, thus further enabling the ability to scale the Digital Image Processing System 800 to meet the demands for a higher input bandwidth, e.g. higher resolution or number of input sources, and larger number of output display regions. For example, a digital image representing a two-dimensional graphics may be digitized and use a linear memory addressing for transferring its video data information in and out of Memory 850, while compressed digital images, e.g. MPEG, may utilize a specific block memory addressing. The Memory Data Path Controller 830 is designed and capable of operating with at least five exemplary different modes (i) progressive scan macro block xy mode, (ii) interlaced scan macro block xy mode, (iii) progressive scan linear xy mode, (iv) interlaced scan linear xy mode, and (v) sequential access mode. Furthermore, the architecture of the Memory Data Path Controller 830 includes a specialized memory organization design that allocates regions of memory locations within the Memory 850 for the exclusive use with certain types of video data information.
Each of the internal modules of Digital Image Processing System 800 generates and outputs, onto the First Memory Data Path 831, memory write or read commands that comprise an operation mode, a base index, and a digital image's location and size information. The location of the digital image within Memory 850 may be determined using a starting memory location that corresponds to the location of a first pixel or origin of the digital image, e.g. top-leftmost pixel. Using the first memory location of a memory region as an origin, then an offset number of memory locations from the first memory location may be used as well to indicate the starting memory location of a desired digital image. The digital image's size information is identified with width, height, or the total number of memory locations used within Memory 850. The Memory Data Path Controller 830 operates and responds to different modes that utilize different location and size information as it may be necessary for the efficient transfer of the video data information from and to Memory 850, as is described later see
The Memory 850 device may be a single memory device or a memory system comprising multiple memory devices or modules, e.g. Memory 851 and Memory 852 as may be desired or necessary by the Digital Image Processing System 800. The Memory 850 is organized into multiple regions, where each region is specified using a starting memory location base address and a predefined number of memory locations. In one preferred embodiment, the Memory 850 is organized into one hundred twenty eight memory regions. In general, the Input Module 810 writes the input digital image video data information into one or more memory regions within Memory 850. The input Module 810 is also capable of splitting the input digital image into smaller regions each of which is stored into a separate memory region within Memory 850. The Output Module 840 reads and processes the video data information as specified using location and size information of a desired digital image from one or more regions within Memory 850. For example, in accordance with a YUV video format standard, the ODM—1 841 module reads the Y-component and UV-component of the video data information from a first and second memory region, respectively. The overall storage capacity of Memory 850 is based on the application and overall processing requirement of video data information of the Digital Image Processing System 800. The storage capacity of each region of Memory 850 is dependent on the resolution, and the type of video data information for a given video format standard being used. Furthermore, various types of memory, e.g. DRAM, SRAM, DDR, etc. . . . , may be used to manufacture Memory 850 as an integrated component within the Digital Image Processing System 800 or as a standalone memory system. In general, the Memory 850 must meet the overall performance and timing requirement of the Input Module 810 and the Output Module 840. The ability to use a particular organization or design of a memory system is influenced by the required overall performance of Memory 850, and the engineering requirement to manage the capacity and location of each of the memory regions within Memory 850. In accordance with one embodiment, each memory region's capacity and location is programmable and can be redefined for a given application's memory usage requirements. The programming of the appropriate registers with a new memory region's size and location within Memory 850, i.e. number of memory words and starting base address, provides a flexible architecture for a scalable and manageable Memory 850 system. The Memory Data Path Controller 830 stores the base address for each memory region in a look up table, where each base address is looked up using the base index embedded within the memory write or read command. Thus, using a base index eliminates the need to reprogram the various modules of Digital Image Processing System 800 with a new set of base addresses, whenever a new organization or reallocation of the memory regions of Memory 850 has taken place. Different storage and lookup means may be used for the base addresses of memory regions of Memory 850 e.g. file registers, programmable registers, Random Access Memory (RAM), or Content Addressable Memory (CAM).
An exemplary Digital Image 900, with a resolution of 48×32, is split into six regions Region—1 911, Region—2 912, Region—3 913, Region—4 914, Region—5 915, and Region—6 916 as shown in
Digital Image 900 has a resolution that corresponds to a total number of 1536 pixels and total number of memory allocation of 36,864 bits (1536 times 24) for a 24 bit RGB or 24,576 bits (1536 times 16) for YUV 4:2:2 video format. In the following examples, certain assumptions are provided as an exemplary mean to describe some of the operation of Digital Image Processing System 800. The video format standard of YUV 4:2:2, 16 bits per pixel is used to store the processed video data information of the captured Digital Image 900. Hence, video data information for one pixel use 8-bits to store the Y-component and another 8-bits for the UV-component. An exemplary 64-bit-word is allocated to each memory location within a first and second region of Memory 850. Hence, the Digital Image 900 would require 384 memory locations to store all of the video data information in accordance with YUV 4:2:2 16 bits video format, organized as 192 memory locations for each of the Y-component and UV-component of the captured Digital Image 900 within the first and second region of Memory 850 respectively. An exemplary first base address of (00001024)10 is used to indicate the actual physical address of the first starting memory location, Memory Location 0 (Row—0, Col—0) 900—800 as shown in
In accordance with one embodiment, the Input Module 810 initiates at least one memory transaction request to transfer and store the Y-component of the video data information of Pixel—0 900—0000 through Pixel—1535 900—1535 in the first memory region within Memory 850. Similarly, Input Module 810 initiates at least one memory transaction to transfer the UV-Component of the video data information of all the pixels of Region—1 911, Region—2 912, Region—3 913, Region—4 914, Region—5 915, and Region—6 916 to be stored in the second memory region within Memory 850. In this example, it is possible for the Y-component of video data information for eight pixels to be stored using a single 64-bit word or one memory location, as described above and shown in
The memory organization of the First Memory Block 911—800 is shown in
Another exemplary memory location is located at the intersection of Row—0 900—300 and Col—5 900—485, i.e. Memory Location 65 (Row 0, Col—5) 900—865, comprising the Y-component of the video data information of the last eight pixels of the first line Row—0 900—300 of the Digital Image 900 as shown in
The First Memory Block 911—800 comprises thirty-two memory locations, zero through thirty-one, as shown in
In accordance with another embodiment, independent memory regions of Memory 850 are used to store each of the video data information of Region—1 911, Region—2 912, Region—3 913, Region—4 914, Region—5 915, and Region—6 916 independently. Similarly, each memory region may be accessed independently from other memory regions, and the retrieved video data information is processed, manipulated, and displayed, using a display region, in accordance with a video effect, resolution, scaling or video format that may be different from what is desired or required by another display region. Various memory organization methods and techniques may be used for how the regions within the Memory 850 are allocated. Accordingly, Memory 850 organization may change with the capacity or type of the physical memory being used. Furthermore, how fast the video data information is to be processed, stored, or retrieved would have an implication on the optimum organization of Memory 850. For example, the video data information of Region—1 911 may be stored sequentially in a linear fashion using Row—0 900—300 through Row—5 900—305, such that the first starting Memory Location 0 900—800 is still located at the intersection of Row—0 900—300 and Col—0 900—480. However, the memory location 31 is now located at the intersection of Row 5 900—305 and Col—1 900—481. In the following paragraphs, exemplary video data information transfers into, and out of Memory 850 is described; please refer to
The Input Module 810 comprises a first and second programmable registers to store a first and second value that corresponds to the Digital Image 900 Width 900—612 and Height 900—622, respectively. The Input Module 810 comprises Input Capture 810—10 to receive and extract the Digital Image 900 video data information via the digital input data path, in accordance with the Digital Image 900 video format, e.g. 24 bit RGB. The Input Capture 810—10 may also be programmed to receive and extract only a predefined region's video data information, e.g. Region—1 911 of the Digital Image 900. Region—1 911 is defined using a starting first pixel location Pixel—0 911—000, its Width 911—612 and Height 911—622 within Digital Image 900. The Input Module 810 comprises Video Formatter 810—20 that processes the captured video data information and generates, in accordance with a second video format e.g. YUV 4:2:2 16 bits, the video data information that corresponds to the Y-component and UV-component of Digital Image 900.
The Input Module 810 further comprises Memory Interface 810—30 comprising a third and fourth programmable registers to store a first and second base index values, e.g. (001)10 and (002)10. The first and second base index values are used as pointers to the first and second base addresses, e.g. (00001024)10 and (00004096)10 of the first and second regions of Memory 850, as described above. This provides the flexibility to reorganize Memory 850 as needed. For example, a particular desired performance of the Digital Image Processing System 800 is satisfied by redefining or reprogramming the base addresses of memory regions, without affecting other memory access requests parameters. The Memory Interface 810—30 generates and outputs onto the First Memory Data Path 831 at least one first memory write command to transfer the Y-component of the video data information of Digital Image 900 starting at Memory Location 0 900—800 of the first region of Memory 850. In this example, the Memory Location 0 is used to store the Y-component of the video data information of the first starting pixel, Pixel—0 900—0000 which is the top-leftmost pixel of the Digital Image 900. A description of memory access request to Memory 850 provides more details about the generation of a memory write or read command or a memory device write or read command.
The first memory write command comprises the first Base Index 1020 value (001)10, and a memory Offset Address 1070 value of (000)10, see
In another embodiment, the first memory write command comprises vertical and horizontal memory position of the first starting memory location of Region—1 911, i.e. Ystart 1030 that corresponds to Row—0 900—300 and Xstart 1040 that corresponds to Col—0 900—480, the intersection of Xstart 1040 and Ystart 1030 corresponds to Memory Location 0 900—800. The first memory write command further comprises the width, Xcount 1050, and height, Ycount 1060, of Region—1 911 in terms of memory words. In this example, the Width 911—612 of Region—1 911 is 16 pixels and corresponds to two 64-bit memory words. The Height 911—622 of Region—1 911 is 16 lines and thus corresponds to sixteen rows of 64-bit memory words, Row—0 911—300 through Row15 911—315, see
In one embodiment, the Memory Interface 810—30 of the Input Module 810 issues multiple memory write commands, as described above, to effect the transfer of all of the Y-component of the video data information of Region—1 911 to the Memory Data Path Controller 830 via the First Memory Data Path 831. Similarly, Memory Interface 810—30 transfers the UV-component of the video data information of Region—1 911 of Digital Image 900 to the second region of Memory 850. The Memory Interface 810—30 generates and outputs onto the First Memory Data Path 831 one, or multiple, second memory write command in accordance with the description of above paragraphs.
The Memory Data Path Controller 830 comprises a First Memory Interface 830—10 that receives the first memory write command and the Y-component of the video data information via the First Memory Data Path 831. The Memory Data Path Controller 830 further comprises logic circuitry to extract from the first memory write command the first Base Index 1020 value (001)10, the memory Offset Address 1070 value of (000)10, and the total number of memory locations, #Words 1080, value of (032)10. In one embodiment, the Memory Data Path Controller 830 further comprises logic circuitry to extract from the first memory write command the vertical and horizontal position of the first starting memory location of Region—1 911, and the width and height information. In this example, width value Xcount 1050=(002)10, the height value Ycount 1060=(016)10, Ystart 1030=(000)10 and Xstart 1040=(001)10, see above.
The Memory Data Path Controller 830 generates the first base address of the first memory region within Memory 850 based on the extracted first base index, Base Index 1020. The Second Memory Interface 830—30 generates the appropriate physical memory address by using the first base address and the memory offset address, Offset Address 1070. The Second Memory Interface 830—30 further generates at least one, or multiple, first memory device write command to complete the transfer of the Y-component of the video data information of Region—1 911 to the first region of Memory 850 via the Second Memory Data Path 832. In this example, the first memory device write command includes the memory address (00001024)10 as the first starting memory location address, where the Y-component of the Memory Location 0 900—800 is to be stored. The content to be stored in the first memory region of Memory 850 in response to the first memory device write command, starting at physical memory address of (00001024)10, corresponds to the Y-component of the video data information of Region—1 911. This includes Memory Location 0 (Row—0, Col—00) 911—800 that corresponds to Pixel Group 0 911—500 through the Memory Location 31 that corresponds to Pixel Group 31 911—531, see Fig. C.
In one embodiment, the Second Memory Interface 830—30 generates at least one, or multiple, first memory device write command to complete the transfer of the Y-component of the video data information of Digital Image 900 to the first region of Memory 850 via the Second Memory Data Path 832. The first memory device write command provides all the necessary control and/or handshake information for Memory 850 to properly decode the first memory device write command and store the Y-component of the video data information of Digital Image 900 in response to the first memory device write command. The stored content in the first memory region of Memory 850, starting at physical memory address of (00001024)10, corresponds to the Y-component of the video data information of Digital Image 900, Memory Location 0 (Row 0, Col—0) 911—800 through the Memory Location 191 (Row—31, Col—5) 911—191. The Memory Data Path Controller 830 further comprises Control 830—20 module to control and manage the transfer of the video data information flow received via the First Memory Data Path 831 and outputted to Memory 850 via the Second Memory Data Path 832. The transfer of the UV-component is described in the following paragraphs.
Similarly, the First Memory Interface 830—10 receives the second memory write command and the UV-component of the video data information, to be transferred in response to the second memory write command, via the First Memory Data Path 831. The logic circuitry extracts at least the second base index information, the starting Memory Location 0, and the total number of memory locations from the second memory write command. In accordance with the same process as described above, the Memory Data Path Controller 830 generates the second base address based on the second base index. Furthermore, The Second Memory Interface 830—30 generates at least one, or multiple, second memory device write command to complete the transfer of the UV-component of the video data information of Digital Image 900 to the second region of Memory 850 via the Second Memory Data Path 832 in response to the second memory write command. The Control 830—20 module controls and manages the transfer of the UV-component of the video data information flow received via the First Memory Data Path 831 and outputted to Memory 850 via the Second Memory Data Path 832. The output display modules retrieve the video data information from Memory 850 for further processing as is described in the following paragraphs.
In one embodiment, the output display module ODM—1 841 is assigned to process the Y-component and the UV-component of the video data information from Region—1 911. The other output display modules that are part of the Output Module 840, e.g. ODM—2 842, is programmed to process the Y-component and the UV-component of the video data information from Region—2 912, and so on for the other ODMs. Each ODM processes one region of the Digital Image 900 and output the results to a dedicated display region of an electronic display panel. The combined picture resulting from all the adjoined display regions, comprising all the processed regions of the Digital Image 900, results in a continuous and seamless display of Digital Image 900 as if it was processed by one output display module. A block diagram of an exemplary output display module is shown in
The first data path of output display module ODM—1 841 comprises a First Input Buffer Control 841—10. The Y-component of the video data information of Region—1 911 is retrieved from Memory 850 using the first data path. Furthermore, the first data path comprises a vertical scaler module VS 841—11 to process and/or scale in the vertical direction the Y-component of the video data information from Region—1 911. An inter-scaler buffer ISB 841—12 provides the necessary buffering and management control of the streamed output of the vertical scaler VS 841—11 and into a horizontal scaler HS 841—13, which processes or scales in the horizontal direction the already vertically scaled Y-component of the video data information from Region—1 911. Since the vertical processing or scaling occurs separately from the horizontal processing or scaling, then the output display module is capable to scale independently the retrieved video data information in either the vertical or the horizontal direction. Similarly, the output display module ODM—1 841 second data path comprises a Second Input Buffer Control 841—20 that is assigned to retrieve the UV-component of the video data information of Region 1 911 from Memory 850. The vertical scaler VS 841—21 processes or scales the UV-component of the video data information and output the vertically processed video data information to inter-scaler buffer ISB 841—22. The horizontal scaler HS 841—23 process or scale the already vertically processed UV-component of the video data information and output the horizontally processed video data information to an Output Formatter 841—50.
The horizontal scalers HS 841—13 and HS 841—23 stream their processed video data information to the Output Formatter 841—50 that further process and recombine the already vertically and horizontally processed or scaled Y-component and the UV-component of the video data information from Region—1 911. The Output Formatter 841—50 streams its output onto the Output Display Data Path 846 via a dedicated channel to a designated display region. The Output Formatter 841—50 processes the video data information of Region—1 911 in accordance with a desired output video format standard as specified by the designated display region, e.g. RGB 4:4:4 24 bits per pixel. The output video format standard can be different from the YUV 4:2:2 16-bits used for storing and processing the video data information of Region—1 911 as has been described above. The output display module ODM—1 841 functions are programmable to independently process or scales the video data information in (i) the vertical direction, (ii) the horizontal direction, and (iii) a designated output video format standard. This provides for a proper display of the final processed video data information of Region—1 911 in accordance with the intended display region of the electronic display panel.
An exemplary Memory Access Request 1000 structure to access the video data information stored in Memory 850, as shown in
The following paragraphs describes the processing of a quadrangle digital image D—1 981 by adding margin pixels to the Region—1 911 as shown in
D—1 Width 981—612=Left Margin 981—626+Width 911—612+Right Margin 981—624 Eq. 5
D—1 Height 981—622=Top Margin 981—623+Height 911—622+Bottom Margin 981—625 Eq. 6
In accordance with one embodiment, The video data information content of each of the pixels within the Top Margin 981—623, forming a new first line of D—1 981, are generated using a copy of the video data information of a corresponding pixel that is directly below it and is at the Top Edge 911—732. For example, the first pixel of the Top Margin 981—623 comprises the video data information of Pixel—0 911—000, and the last pixel of the Top Margin 981—623 comprises the video data information of Pixel—015 911—015.
Similarly, the video data information content of each of the pixels within the Bottom Margin 981—625, forming a new last line of D—1 981, are generated using a copy of the video data information of a corresponding pixel that is directly above it and is at the Bottom Edge 911—752. For example, the first pixel of the Bottom Margin 981—625 comprises the video data information of Pixel—240 911—240, and the last pixel of the Bottom Margin 981—625 comprises the video data information of Pixel—255 911—255.
In accordance with one embodiment, the video data information content of each of the pixels within the Left Margin 981—626, forming a new first column of pixels for D—1 981, is generated using a copy of the video data information of a corresponding pixel that is directly to the right of it. For example, the top pixel of the Left Margin 981—626 comprises the video data information of the first pixel of the Top Margin 981—623. The bottom pixel of the Left Margin 981—626 comprises the video data information of the first pixel of the Bottom Margin 981—625. The other pixels of the Left Margin 981—626 comprise the video data information of the pixels of the Left Edge 911—762.
Similarly, the video data information content of each of the pixels within the Right Margin 981—624, forming a new last column of pixels for D—1 981, is generated using a copy of the video data information of a corresponding pixel that is directly to the left of it. For example, the top pixel of the Right Margin 981—624 comprises the video data information of the last pixel of the Top Margin 981—623. The bottom pixel of the Right Margin 981—624 comprises the video data information of the last pixel of the Bottom Margin 981—625. The other pixels of Right Margin 981—624 comprise the video data information of the pixels of the Right Edge 911—742.
In the following paragraphs, the first data path of an output display module, e.g. ODM—1 841, is used to describe an exemplary vertical and horizontal processing of the video data information for Region—1 911 as shown in
In accordance with one embodiment, the generation of the video data information of the pixels of the Top Margin 981—623 is done on the fly. When the first line Row—0 911—300, as shown in
Similarly, the generation of the video data information of the Bottom Margin 981—625 is done on the fly when the last line of Region—1 911, namely Row—15 911—315 see
Other processes or methods may be used as well to effect the generation of the pixels of the Top Margin 981—623 and Bottom Margin 981—625 from pixels within Region—1 911 or neighboring regions, e.g. Region—2 912. Variety of methods as described above may be employed to generate the pixels of the Top Margin 981—623 and Bottom Margin 981—625. Furthermore, various types of data filtering or data interpolation can be used to process the video data information of the designated digital image and generate on the fly the video data information for the margin pixels. For example, video data information of a specific color can be used to generate the video data information of the pixels of the Top Margin 981—623 and Bottom Margin 981—625.
In this example, D—1 Height 981—622 comprises 18 lines and can be computed using Eq. 6. For example, the sum of (i) the Top Margin 981—623 that comprises one line, (ii) Row—0 911—300 through Row—15 911—315 comprising 16 lines, and (iii) the Bottom Margin 981—625 that comprises one line. At this stage, after the processing in the vertical direction is completed, the width of vertically processed lines stored in the inter-scaler buffer ISB 841—12 corresponds to 16 pixels, and is the same as Width 911—612, as shown in
The vertical scaler, VS 841—11, operates using a group of two or more lines at a time to generate one vertically processed line that is outputted to the inter-scaler buffer, ISB 841—12, for temporary storage. The number of lines that are used to produce one vertically processed line depends on the type of scaling process methodology and the number of coefficients required to provide a desired accuracy for a given scaling process and/or a given scaling ratio for the intended quality level of the scaled output image. In this example, the video data information of the pixels of two vertically adjacent lines are used to generate the video data information for the vertically processed line. In accordance with one embodiment, eight vertically adjacent lines are used to generate one vertically processed line that is outputted to the inter-scaler buffer, ISB 841—12, for temporary storage. In accordance with one embodiment, four vertically adjacent lines are used to generate one vertically processed line that is outputted to the inter-scaler buffer, ISB 841—12, for temporary storage. If the vertical scaling ratio is equal to two, then the total number of the vertically processed lines outputted to the inter-scaler buffer ISB 841—12 is twice that of D—1 Height 981—622. In this example, the vertical scaler VS 841—11 would output thirty-six vertically processed lines, which is twice that of the D—1 Height 981—622 comprising eighteen lines, sixteen lines of which are from Region—1 911 and one line each from the Top Margin 981—623 and Bottom Margin 981—625.
An exemplary processing of two vertically adjacent lines to produce one new vertically processed line is described as follows; please refer to
The vertical scaling process now repeats using the second line of D—1 981 that corresponds to Row—1 981—301, which is the same as Row—0 911—300, and the third line of D—1 981 that corresponds to Row—2 981—302. Thus, the second vertically processed line is generated and outputted to the inter-scaler buffer ISB 841—12 for temporary storage. The vertical scaling process continues until the eighteenth vertically processed line of the vertically scaled output image is generated and outputted to the inter-scaler buffer ISB 841—12 for temporary storage. As the vertical scaling process operation is being completed, then the video data information of the pixels of the topmost lines of the First Input Buffer Control 841—10 are no longer needed for the vertical scaling process operation of the current digital image. Therefore, the topmost lines, e.g. Row—0 981—300, Row—1 981—301, etc. . . . , may now be discarded, and in accordance with the design requirement one or more lines may be discarded at a time. A first-in-first-out FIFO buffer may be used within the First Input Buffer Control 841—10 to store the video data information. As the topmost line or lines are no longer needed, then the First Input Buffer Control 841—10 may initiate a memory read operation to retrieve a single new line or multiple lines of video data information to be read out of memory and into the First Input Buffer Control 841—10 overwriting the line or lines to be discarded. The First Input Buffer Control 841—10 maintains its FIFO full so that the vertical scaling process progress continuously as needed or required by the output display module ODM—1 841 and in accordance with the desired performance of the Digital Image Processing System 800. Furthermore, the topmost line or lines of the First Input Buffer Control 841—10 may be kept for a certain amount of time, long enough to perform a recall or a video effect operation as may be desired or necessary. Upon the initiation of the horizontal scaling process for the first vertically processed line, the horizontal scaler HS 841—13 retrieve the first vertically processed line from inter-scaler buffer ISB 841—12. The left and right margin pixels are generated for all of the vertically processed lines as will be described in the following paragraphs.
In accordance with one embodiment, the generation of the video data information of the pixels of the Left Margin 981—626 and the Right Margin 981—624 is done on the fly. For example, as the video data information of the pixels of the vertically processed line is being streamed into the horizontal scaler HS 841—13 from the inter-scaler buffer ISB 841—12 temporary storage. Other means for generating the Left Margin 981—626 and the Right Margin 981—624 pixels may be used as well, as described earlier in this disclosure. In this example, the left margin pixel and the right margin pixel are generated using a copy of the first pixel and the last pixel of each vertically processed line, respectively. For example, the first pixel and the last pixel of the first vertically processed line, which is stored in the inter-scaler buffer ISB 841—12 temporary storage, are replicated to generate the top pixel Pixel—0 981—000 of the Left Margin 981—626 and the top pixel Pixel—117 981—017 of the Right Margin 981—624, respectively. In accordance with one embodiment, upon the initiation of the horizontal scaling process, the new vertically processed line including the left and right margin pixels is streamed to the horizontal scaler HS 841—13.
The horizontal scaling process is similar to the vertical scaling process as described above and utilize two or more horizontally adjacent pixel to generate one horizontally processed pixel. In this example, two horizontally adjacent pixels are used to generate one horizontally processed pixel. This horizontally processed pixel is streamed to the Output Formatter 841—50 for further processing. The horizontal processing continues until all of the horizontally processed pixels are generated in accordance with the desired horizontal scaling ratio. Similarly, the process continues for all the vertically processed lines, including the left and right margin pixels, and all the vertically and horizontally processed lines are streamed to the Output Formatter 841—50 in the same way as for the first vertically processed line. Since we are using a scaling ratio of one in this example, the vertically and horizontally processed D—1 981 comprises eighteen rows, Row—0 981—300 through Row—17 981—317 and each row having a width that is equal to eighteen pixels, Width 981—612 see
The second data path of the output display module ODM—1 841 independently retrieves, and processes video data information in the vertical or the horizontal directions. In this example, the UV-component of D—1 981 is independently retrieved using the Second Input Buffer Control 841—20. The vertical and horizontal processing is performed using a vertical scaler VS 841—21, an inter-scaler buffer ISB 841—22, and a horizontal scaler HS 841—23. In a similar fashion as described above for the first data path, the horizontal scaler HS 841—23 streams the vertically and horizontally processed video data information to the output formatter 841—50.
The Output Formatter 841—50 further processes and appropriately synchronizes and combines the vertically and horizontally processed Y-component and UV-component of D—1 981 to generate and output onto the Output Display Data Path 846 the desired output image in accordance with an output video format standard that is compatible with the destination display region. The Output Formatter 841—50 appropriately crops the combined vertically and horizontally processed pixels that are the result of the appended Top Margin 981—623, Bottom Margin 981—625, Left Margin 981—626, and Right Margin 981—624. The combined vertically and horizontally processed pixels, which are used in the generation of the desired output image that corresponds to Region—1 911, when displayed alongside the other vertically and horizontally processed pixels of neighboring regions will result in a contiguous and a full display a vertically and horizontally processed Digital Image 900. The Output Formatter 841—50 streams or outputs each synchronized and combined vertically and horizontally processed line onto the Output Display Data Path 846, the Output Formatter 841—50 uses a data enable signal to indicate to an electronic display device to discard or blank one or more pixels for each line. In some instances, an entire line may be blanked. Therefore, in this example with a scaling ratio of one, the desired processed output image to be displayed that corresponds to Region—1 911 comprises sixteen lines, where each line comprises sixteen pixels. The following paragraphs will describe a slightly different method to process or scale Region—1 911.
In accordance with one embodiment, the vertical and horizontal processing of Region—1 911 of the Digital Image 900 is accomplished by defining a quadrangle digital image D—2 982 that includes all of the pixels of Region—1 911 and a right and bottom margin pixels from neighboring regions, as shown in
Now referring to
Now referring to
In accordance with one embodiment, a quadrangle digital image D—3 983 comprises the pixels of Region—2 912, and pixels from neighboring left, bottom, and right regions of Digital Image 900, as shown in
In accordance with one embodiment, the quadrangle digital image D—3 983 comprises the pixels of Region—2 912, and pixels from neighboring left, bottom, and right regions of Digital Image 900, as shown in
The Digital Image 900 is split into six regions and each region may be further split into four sub-regions as shown in
In this example, a memory read command is initiated by output display module ODM—1 841 to retrieve the Second Memory Block 912—30—800 that comprises the Y-component of the video data information of the pixels of Region—2—3 912—30. The First Input Buffer Control 841—10 produces a memory read command comprising a Base Index 1020 to read the video data information out of Memory 850. The Base Index 1020 value corresponds to the first starting memory location of the first region of Memory 850, and in this example equals (001)10. The Region 2—3 912—30 comprises eight memory locations, namely: 48, 50, 52, 54, 56, 58, 60, and 62. The vertical and horizontal origin of the Region—2—3 912—30, comprising the top-leftmost pixel of Region—2—3 912—30, is identified using Ystart 1030 and Xstart 1040 values, respectively. In this example, the location information of the vertical and horizontal origin is given by Ystart 1030=(008)10 that corresponds to the ninth row (Row—8), and Xstart 1040=(002)10 that corresponds to the third column (Col—2), see Memory Location 48 (Row—8, Col—2) 900—848. Furthermore, the size information is given by Xcount 1060=(001)10 that corresponds to one memory location in width or the horizontal direction, and Ycount 1050=(008)10 that corresponds to eight memory locations in height or the vertical direction. Therefore, the exemplary memory read command, in accordance with the command structure of
Now referring to
In this example, a memory read command is initiated by the output display module ODM—1 841 to retrieve the Third Memory Block 991—123—800, comprising the Y-component of the video data information of the pixels of Region—991—123 991—123—900. The First Input Buffer Control 841—10 produces a memory read command comprising a Base Index 1020 to read the video data information out of Memory 850. The Base Index 1020 value corresponds to the first starting memory location of the first region of Memory 850, and in this example equals (001)10. The vertical and horizontal origin of the Region—991—123 991—123—900, comprising the top-leftmost pixel of Region—991—123, is identified using Ystart 1030 and Xstart 1040 values, respectively. In this example, the location information is given by Ystart 1030=(000)10 that corresponds to the first row (Row—0), and Xstart 1040=(000)10 that corresponds to the first column (Col—0). Furthermore, the size information is given by Xcount 1060=(006)10 that corresponds to six memory location in width or the horizontal direction, and Ycount 1050=(001)10 that corresponds to one memory location in height or the vertical direction, i.e. one line of the Digital Image 900. Therefore, the exemplary memory read command, in accordance with the structure of
Now referring to
Now referring to
The exemplary embodiments as described in this disclosure are used interchangeably to develop a certain and unique Digital Image Processing System 800 that meets a widely varying input digital image processing requirements, including various techniques to process and split an input digital image. Furthermore, various embodiments provide the ability to customize the Digital Image Processing System 800 to meet an ever-changing electronic display panels' size and specification.
Claims
1. A method for processing video data information of an input image, comprising:
- retrieving from a memory buffer a first video data information that corresponds to at least a first portion of the input image;
- generating a second video data information that corresponds to at least one pixel of one of a top margin and a bottom margin;
- generating a third video data information that corresponds to at least one pixel of one of a right margin and a left margin;
- storing the first and second video data information using a first buffer;
- processing the first and second video data information to produce a first processed video data information;
- outputting the first processed video data information to a second buffer;
- storing the third video data information and the first processed video data information using the second buffer; and
- processing the first processed video data information and the third video data information to produce a second processed video data information, wherein the second processed video data information corresponds to a first number of pixels.
2. The method of claim 1, further comprising:
- generating a first data enable signal information to control displaying of a second number of pixels, wherein the second number of pixels is less than or equal to the first number of pixels; and
- outputting the second processed video data information and the first data enable signal information to an electronic display.
3. The method of claim 2, further comprising:
- displaying, using a first display region of the electronic display, the second number of pixels using the second processed video data information and the first data enable signal information.
4. The method of claim 1, further comprising:
- determining a first location information of a first pixel of a quadrangle digital image, wherein the first location information comprises at least one of a first base index and information corresponding to a first memory location of the memory buffer.
5. The method of claim 4, wherein the first video data information is retrieved from the memory buffer using the first location information.
6. The method of claim 4, wherein the quadrangle digital image comprises at least the first portion of the input image, the top margin, the bottom margin, the left margin, and the right margin.
7. The method of claim 1, wherein any one of the top margin, the bottom margin, the left margin, and the right margin comprises video data information of at least one pixel that is generated using at least one of a video data information corresponding to a pixel of the first portion of the input image, a video data information corresponding to a pixel of the input image, and a video data information corresponding to a color.
8. The method of claim 1, wherein the processing of the first and second video data information to produce a first processed video data information includes processing in the vertical direction of at least one pixel of the first portion of the input image and at least one pixel of one of the top margin and the bottom margin.
9. The method of claim 8, wherein the processing in the vertical direction includes scaling in the vertical direction using a scaling value.
10. The method of claim 1, wherein the processing of the first processed video data information and the third video data information to produce a second processed video data information includes processing in the horizontal direction of at least one pixel of the first portion of the input image and at least one pixel of one of the left margin and the right margin.
11. The method of claim 8, wherein processing in the horizontal direction includes scaling in the horizontal direction using a scaling value.
12. A method for processing video data information of an input image, comprising:
- retrieving from a memory buffer a first video data information that corresponds to at least a first portion of the input image;
- generating a second video data information that corresponds to at least one pixel of one of a top margin and a bottom margin;
- generating a third video data information that corresponds to at least one pixel of one of a right margin and a left margin;
- processing the first and second video data information to produce a first processed video data information;
- processing the first processed video data information and the third video data information to produce a second processed video data information, wherein the second processed video data information corresponds to a first number of pixels; and
- storing the second processed video data information using a first buffer.
13. The method of claim 12, further comprising:
- generating a first data enable signal information to control displaying of a second number of pixels, wherein the second number of pixels is less than or equal to the first number of pixels; and
- outputting the second processed video data information and the first data enable signal information to an electronic display.
14. The method of claim 13, further comprising:
- displaying, using a first display region of the electronic display, the second number of pixels using the second processed video data information and the first data enable signal information.
15. The method of claim 12, wherein any one of the top margin, the bottom margin, the left margin, and the right margin comprises video data information of at least one pixel that is generated using at least one of a video data information corresponding to a pixel of the first portion of the input image, a video data information corresponding to a pixel of the input image, and a video data information corresponding to a color.
16. The method of claim 12, wherein any one of the first processed video data information and the second processed video data information is produced using any one of a processing in the vertical direction of at least one pixel of the first portion of the input image and at least one pixel of one of the top margin and the bottom margin, and a processing in the horizontal direction of at least one pixel of the first portion of the input image and at least one pixel of one of the left margin and the right margin.
17. The method of claim 16, wherein the processing in the horizontal direction includes scaling in the horizontal direction using a first scaling value, the processing in the vertical direction includes scaling in the vertical direction using a second scaling value, and the first scaling value corresponds to a value that is different or same as the second scaling value.
18. A system to process video data information of an input image comprising:
- a video memory buffer;
- a first memory buffer;
- a second memory buffer; and
- a first circuit that is configured to: (i) retrieve from the video memory buffer a first video data information that corresponds to at least a first portion of the input image, (ii) generate a second video data information that corresponds to at least one pixel of one of a top margin and a bottom margin, (iii) store the first and second video data information using the first memory buffer, (iv) process the first and second video data information to produce a first processed video data information, (v) generate a third video data information that corresponds to at least one pixel of one of a right margin and a left margin, (vi) store the third video data information and the first processed video data information using the second buffer, and (vii) process the first processed video data information and the third video data information to produce a second processed video data information, wherein the second processed video data information corresponds to a first number of pixels.
19. The system of claim 18, further comprising:
- a second circuit that is configured to (i) generate a data enable signal to control display of a second number of pixels, and (ii) output the second processed video data information and the data enable signal, wherein the second number of pixels is less than or equal to the first number of pixels.
20. The system of claim 19, further comprising:
- a display device having at least a first and second display regions, the display device includes a third circuit that is configured to (i) receive the second processed video data information and the data enable signal, and (ii) display the second number of pixels using a first region of the display device, wherein the third circuit controls the display of the second number of pixels using the data enable signal.
21. The system of claim 18, wherein any one of the top margin, the bottom margin, the left margin, and the right margin comprises video data information of at least one pixel that is generated using at least one of a video data information corresponding to a pixel of the first portion of the input image, a video data information corresponding to a pixel of the input image, and a video data information corresponding to a color.
22. The system of claim 18, wherein the first circuit is further configured to produce the first processed video data information by processing in the vertical direction at least one pixel of the first portion of the input image and at least one pixel of one of the top margin and the bottom margin.
23. The system of claim 18, wherein the first circuit is further configured to scale, in the vertical direction, the first processed video data information using a scaling value.
24. The system of claim 18, wherein the first circuit is further configured to produce the second processed video data information by processing in the horizontal direction at least one pixel of the first portion of the input image and at least one pixel of one of the left margin and the right margin.
25. The system of claim 18, wherein the first circuit is further configured to scale, in the horizontal direction, the second processed video data information using a scaling value.
26. The system of claim 18, wherein the first circuit comprises at least one of a logic circuit, a central processing unit (CPU), a software, and a firmware.
27. The system of claim 26, wherein the first circuit is programmed to perform one or more of the first circuit configurations (i) to (vii).
Type: Application
Filed: Feb 11, 2013
Publication Date: Mar 6, 2014
Applicant: Marseille Networks, Inc. (Santa Clara, CA)
Inventors: Brian J. Anderson , Emmanuel Hislen (Alameda, CA)
Application Number: 13/763,735
International Classification: G06K 9/36 (20060101); G06T 3/40 (20060101);
