JOURNALING DATABASE CHANGES USING MINIMIZED JOURNAL ENTRIES THAT MAY BE OUTPUT IN HUMAN-READABLE FORM
A minimized journaling mechanism stores minimized journal data in a format that allows for display and outputting the journal data in human-readable form. When a change to a record occurs, instead of writing only the changed bytes, all of the bytes in each field that changed are written to the journal, along with all of the bytes in each field selected to be always journaled. A default object is created with default data in all of the fields. When the journal entry needs to be output in human-readable form, the default object is read, and the minimized journal entry is then overlaid on the default object. The result is an object that contains default data in all non-selected fields that were not changed, with the journal data in all fields and that did change and in all fields that were selected to always be journaled.
Latest IBM Patents:
- INTERACTIVE DATASET EXPLORATION AND PREPROCESSING
- NETWORK SECURITY ASSESSMENT BASED UPON IDENTIFICATION OF AN ADVERSARY
- NON-LINEAR APPROXIMATION ROBUST TO INPUT RANGE OF HOMOMORPHIC ENCRYPTION ANALYTICS
- Back-side memory element with local memory select transistor
- Injection molded solder head with improved sealing performance
This patent application is a continuation of a patent application with the same title, U.S. Ser. No. 11/291,582 filed on Dec. 1, 2005, which is incorporated herein by reference.
BACKGROUND OF THE INVENTION1. Technical Field
This invention generally relates to computer systems, and more specifically relates to journaling changes in a database.
2. Background Art
Database systems have been developed that allow a computer to store a large amount of information in a way that allows a user to search for and retrieve specific information in the database. For example, an insurance company may have a database that includes all of its policy holders and their current account information, including payment history, premium amount, policy number, policy type, exclusions to coverage, etc. A database system allows the insurance company to retrieve the account information for a single policy holder among the thousands and perhaps millions of policy holders in its database.
Database systems typically include a database manager that assures the integrity of the database as changes to the database are made. The database manager typically records changes to a database in a journal. The journal allows changes to the database to be rolled back if needed. In addition, a journal may be used to replicate a database by applying the journaled changes to a remote database in the same order the journaled changes were made to the original database.
Most database managers known in the art record all bytes in a record into a log when a change is made to the record, including bytes in the record that do not change. Because recording data that has not changed takes valuable space and time in a database journal, a newer concept known as minimized data journaling was introduced. With a minimized data journal, only the changed bytes of a record are recorded without respect to field boundaries, which assumes that the bytes in the record that are not represented in the journal entry did not change. A minimized data journal is generally likely to achieve significantly better performance than a full data journal. Note, however, that this increase in performance comes at a usability cost.
Because a journal entry in a minimized data journal only includes the bytes that changed, the resulting journal entry is in a compressed format that may no longer be easily read, understood, and recognized by a human or auditing program. While this is not a concern in some settings, it may be a significant concern in other settings that require auditing, debugging, or human viewing of database changes. To perform an audit, a human auditor or auditing program typically compares the full image of the record before the change to a full image of the record after the change. The human auditor or auditing program thus expects to see the full record in performing the audit. Without a full record image, sensing the identity of the record (e.g., John Doe's bank account #12345), lining up the matching fields among the set of data collected, and recognizing field boundaries is made very difficult and does not constitute a friendly human interface. Because most minimized data journal entries do not contain the full record, nor even full fields, the resulting encoded data is not human-readable. For the purposes herein, the term “human-readable” means a format that does not portray an internal encoded representation of the changes, but rather a representation of the changes in full record layout with no partially represented fields. The data represented in this manner allows a human auditor or auditing program to easily discern the change that took place and is displayed in a manner consistent with using full journal data. For this reason, minimized data journals have generally not been used in environments that require auditing. As a result, environments that require auditing or occasional debugging based on the journal have not benefitted from the performance increase provided by minimized data journals. Without a way to provide minimized journal data that may be output in human-readable form, the computer industry will be left with the undesirable choice between higher performance systems that use minimized journal data but cannot be easily audited, or lower performance systems that use full journal data that can be audited. Neither of these choices is desirable when both high performance and auditing is required.
DISCLOSURE OF INVENTIONAccording to the preferred embodiments, a minimized journaling mechanism stores minimized journal data in a format that allows for display and outputting the journal data in human-readable form that may be easily read and recognized by an auditor. When a change to a record occurs, instead of writing only the changed bytes, all of the bytes in each field that changed are written to the journal, along with all of the bytes in each field selected to be always journaled. Fields in neither of these categories are not captured. A default object is created with default data in all of the fields. When the journal entry needs to be output in human-readable form, the default object is read, and the minimized journal entry is then overlaid on the default object. The result is an object that contains default data in all non-selected fields that were not changed, with the journal data in all fields and that did change and in all fields that were selected to always be journaled. In this manner the preferred embodiments provide the benefits of minimized journal data while still providing for the ability to audit from the journal.
The foregoing and other features and advantages of the invention will be apparent from the following more particular description of preferred embodiments of the invention, as illustrated in the accompanying drawings.
The preferred embodiments of the present invention will hereinafter be described in conjunction with the appended drawings, where like designations denote like elements, and:
The preferred embodiments provide minimized journal data in human-readable form. This is accomplished by writing all data in each field that was changed, and in each field that is specifically selected for journaling, to the journal. A default object with default values in each field is also saved to the journal. When a journal entry needs to be output in human-readable form, the corresponding default object is read, the minimized journal entry is read, and the data from the minimized journal entry is overlaid on the default object. The result is a record that contains all the original fields, with data in all fields that changed and in all fields that were selected for journaling, and an indication of default data in the fields that did not change and were not selected for journaling. This record is human-readable, even though it contains minimized journal data. The preferred embodiments thus provide the best of both worlds: significantly improved performance due to minimizing journal data collection while providing the ability to audit the journal data due to the ability to output the minimized journal data in a human-readable form.
Referring to
Main memory 120 in accordance with the preferred embodiments contains data 121, an operating system 122, a database 123, a minimized journaling mechanism 125, and a journal 126. Data 121 represents any data that serves as input to or output from any program in computer system 100. Operating system 122 is a multitasking operating system known in the industry as i5/OS; however, those skilled in the art will appreciate that the spirit and scope of the present invention is not limited to any one operating system. Database 123 is any suitable database, whether currently known or developed in the future. Database 123 comprises one or more records 124 that contain fields of data. Minimized journaling mechanism 125 suitably generates journal entries in the journal 126. The individual entries in
Computer system 100 utilizes well known virtual addressing mechanisms that allow the programs of computer system 100 to behave as if they only have access to a large, single storage entity instead of access to multiple, smaller storage entities such as main memory 120 and DASD device 155. Therefore, while data 121, operating system 122, database 123, minimized journaling mechanism 125, and journal 126 are shown to reside in main memory 120, those skilled in the art will recognize that these items are not necessarily all completely contained in main memory 120 at the same time. It should also be noted that the term “memory” is used herein generically to refer to the entire virtual memory of computer system 100, and may include the virtual memory of other computer systems coupled to computer system 100.
Processor 110 may be constructed from one or more microprocessors and/or integrated circuits. Processor 110 executes program instructions stored in main memory 120. Main memory 120 stores programs and data that processor 110 may access. When computer system 100 starts up, processor 110 initially executes the program instructions that make up operating system 122. Operating system 122 is a sophisticated program that manages the resources of computer system 100. Some of these resources are processor 110, main memory 120, mass storage interface 130, display interface 140, network interface 150, and system bus 160.
Although computer system 100 is shown to contain only a single processor and a single system bus, those skilled in the art will appreciate that the present invention may be practiced using a computer system that has multiple processors and/or multiple buses. In addition, the interfaces that are used in the preferred embodiments each include separate, fully programmed microprocessors that are used to off-load compute-intensive processing from processor 110. However, those skilled in the art will appreciate that the present invention applies equally to computer systems that simply use I/O adapters to perform similar functions.
Display interface 140 is used to directly connect one or more displays 165 to computer system 100. These displays 165, which may be non-intelligent (i.e., dumb) terminals or fully programmable workstations, are used to allow system administrators and users to communicate with computer system 100. Note, however, that while display interface 140 is provided to support communication with one or more displays 165, computer system 100 does not necessarily require a display 165, because all needed interaction with users and other processes may occur via network interface 150.
Network interface 150 is used to connect other computer systems and/or workstations (e.g., 175 in
At this point, it is important to note that while the present invention has been and will continue to be described in the context of a fully functional computer system, those skilled in the art will appreciate that the present invention is capable of being distributed as a program product in a variety of forms, and that the present invention applies equally regardless of the particular type of computer-readable signal bearing media used to actually carry out the distribution. Examples of suitable computer-readable signal bearing media include: recordable type media such as floppy disks and CD RW (e.g., 195 of
A simple example is now presented to compare and contrast the prior art methods for journaling and the method of the preferred embodiments, which is preferably performed by the minimized journaling mechanism 125 in
We now assume an update operation needs to be performed on Record 2 in
There are different ways in the art for the change in Record 2 from
One advantage of the prior art full journaling shown in method 500 in
Due to the significant performance penalty that results from full journaling, a different method was developed to only write the bytes in a record that changed to the journal. This is known in the art as minimized data journaling, which is done without regard to field boundaries. Referring to
The preferred embodiments solve the problem of not being able to audit minimized journal data while still providing nearly all of the advantages of minimized data journaling. In the preferred embodiments, a minimized journal entry 1010 shown in
When the minimized journal entry 1010 is stored to the journal, the minimized journal mechanism (e.g., 125 in
As shown in
Referring to
Method 1300 in
Referring to
The preferred embodiments provide significant advantages over the prior art. By storing only slightly more data in the journal than the prior art does during minimized data journaling, almost all of the performance benefits of minimized data journaling are realized in the preferred embodiments, while still providing an output in human-readable form that allows for auditing the journal. The result is the best of both worlds—high performance from minimizing the data stored, while still allowing auditing.
Not all users of the stored journal information will need full auditing and human recognizable capabilities. One of the advantages of this approach is that both uses are readily available. That is, just because the record change was captured in a minimized entry representation does not mandate that it be re-expanded into a human-readable representation at the time it is displayed to a user. The preferred embodiments disclose storing a minimized journal entry in the journal in a first format, and outputting the minimized journal entry in either this first internal format or human-readable form in a second format. Providing these two different formats provides great flexibility. For example, the first format could be used to replicate the database to a remote system thereby assuring reduced transmission volumes, while the second human-readable format can be used for auditing or debugging purposes. The two different formats allow the minimized journal mechanism of the preferred embodiments to serve both the human auditor and the machine-driven replay for replication with equal ease without mandating that two different instances be captured and stored.
One advantage of the preferred embodiments is that the journal alone contains all of the necessary data to display the human-readable form of the minimized journal entry. That is, the related database table being audited or debugged need not even exist on the system used by the auditor to view the journal entries. This is accomplished by storing the default objects in the journal as well. This makes the journal independently able to display the journal entries. As a consequence, a journal may be saved to media or sent remotely to a secondary system with the assurance that the journal contains the complete audit record. This is especially attractive in an auditing environment where the journal can be easily archived or transferred to a second system for analysis.
One skilled in the art will appreciate that many variations are possible within the scope of the present invention. Thus, while the invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that these and other changes in form and details may be made therein without departing from the spirit and scope of the invention.
Claims
1. A computer-implemented method for storing a change to a record in a database in an entry in a journal, the method comprising the steps of:
- generating the entry, the entry including all data from all fields in the record that changed, the entry including all data from any selected fields that are always to be journaled, the entry excluding all data from all non-selected fields in the record that did not change, the entry including a collected fields bitmap that indicates which fields in the entry were collected and which fields were not collected in the entry and a changed fields bitmap that indicates which fields in the entry were changed and which fields in the entry were not changed; and
- storing the entry in the journal.
2. The method of claim 1 wherein the selected fields are selected by a user.
3. The method of claim 1 wherein the selected fields are selected by specifying a type, wherein all fields of the specified type are included in the selected fields.
4. The method of claim 1 further comprising the steps of:
- determining whether a default object corresponding to the entry is already stored in the journal; and
- if the default object corresponding to the entry is not already stored in the journal, storing the default object corresponding to the entry in the journal.
5. The method of claim 4 wherein the default object includes default data for each field in the object.
6. The method of claim 4 further comprising the step of outputting the entry in human-readable form by performing the steps of:
- reading the default object;
- reading the entry;
- overlaying the data from the entry onto the default object; and
- outputting the overlaid object.
7. The method of claim 6 further comprising the step of displaying the human-readable form to a user, wherein data for all selected fields and all fields that changed are displayed to the user, and wherein the default data is represented as data that did not change in the display of the human-readable form.
8. A computer-implemented method for storing a change to a record in a database in an entry in a journal and outputting the entry in the journal in human-readable form, the method comprising the steps of: displaying the human-readable form of the overlaid object to a user, wherein data for all selected fields and all fields that changed are displayed to the user, and wherein the default data is represented as data that did not change in the display of the human-readable form.
- generating the entry, the entry including all data from all fields in the record that changed, the entry including all data from any selected fields that are always to be journaled, wherein the selected fields are selected by a user by specifying a type, wherein all fields of the specified type are included in the selected fields, the entry excluding all data from all non-selected fields in the record that did not change, the entry including a collected fields bitmap that indicates which fields in the entry were collected and which fields were not collected in the entry and a changed fields bitmap that indicates which fields in the entry were changed and which fields in the entry were not changed;
- storing the entry in the journal;
- determining whether a default object corresponding to the entry is already stored in the journal, wherein the default object includes default data for each field in the object;
- if the default object corresponding to the entry is not already stored in the journal, storing the default object corresponding to the entry in the journal;
- outputting the entry in human-readable form by performing the steps of: reading the default object; reading the entry; overlaying the data from the entry onto the default object; outputting the overlaid object; and
Type: Application
Filed: Apr 24, 2008
Publication Date: Aug 21, 2008
Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION (Armonk, NY)
Inventors: Adam Thomas Stallman (Rochester, MN), Larry William Youngren (Rochester, MN)
Application Number: 12/109,051
International Classification: G06F 17/30 (20060101);