Application Risk and Control Assessment

Abstract

Systems and methods are disclosed for assessing risks that may be associated with an application and assessing controls that may be implemented to mitigate the risks associated with the application. The systems and methods may include identifying risk data that may be used at least partially to calculate a risk score. The systems and methods may also include identifying control data that may be used at least partially to calculate a control score. The risk score and the control score may be compared to one another. An assessment of the risks and controls application may be performed at least partially based on the risk score and the control score. A threshold value may be set for the risk score and/or the control score that are compared and used to assess an application.

Description

CROSS REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No. 11/849,245 filed on Aug. 31, 2007, which is a continuation-in-part of U.S. application Ser. No. 11/279,301 filed on Apr. 11, 2006, now U.S. Pat. No. 8,135,605, which are herein incorporated by reference in their entirety.

FIELD OF THE INVENTION

Aspects of the present invention relate to collecting, analyzing and displaying information about the risks and controls associated with a line of business. More specifically, in one aspect, it is directed to such a tool configured to study various portfolios of software applications used by an organization.

BACKGROUND

Typically, organizations have a vision, goals, and specific objectives aimed at achieving the goals and realizing the vision. To achieve goals and realize a vision, an organization must overcome risks that jeopardize the achievement of the organizational objectives, goals, or vision and create uncertainty that the desired results will be achieved. Organizations must identify risks that put their objectives in jeopardy and deploy controls to reduce the risks to acceptable levels.

An organization's management is often hampered by the lack of consistent methods for (a) identifying and measuring the risks associated with each endeavor, (b) digesting information about the degree to which controls have been implemented to combat those risks, and (c) linking the risks and controls to accountability within the organization. Poor management of the organization results in an inability to set priorities and a failure to achieve an optimal allocation of resources toward risks and controls across the entire enterprise. A failure to properly assess an organization's risks and controls exposes the organization to unknown or unidentified risks will prevent the achievement of the organization's objectives. Resources will be wasted on inefficient and/or ineffective assessment of risks and controls.

SUMMARY OF THE INVENTION

In one aspect, the invention is a method of assessing an application comprising identifying risk data that relates to an application and identifying control data that relates to the application. A risk score may be generated that is based at least in part on the risk data. A control score may be generated that is based at least in part on the control data. The application may be assessed based at least in part on the risk score and the control score.

In another aspect of the invention, an apparatus for assessing an application is disclosed comprising a memory and a processor. The memory may store a plurality of modules that comprise computer-executable instructions. The plurality of modules may include a risk data module, a risk score module, a control data module, a control score module, and a comparison module. The risk data module may be configured to store risk data relating to the application. The risk score module may be configured to generate a risk score that is based at least in part on the risk data. The control data module may be configured to store control data that relates to the application. The control score module may be configured to generate a control score that may be based at least in part on the control data. The comparison module may be configured to compare the risk score and the control score. The processor may be configured to execute the computer-readable instructions in the plurality of modules and may assess the application based at least in part on the risk score and the control score.

In another aspect of the invention, a computer-readable medium comprises computer-executable instructions to perform a method. The method may comprise identifying risk data relating to an application, generating a risk score based at least in part on the risk data, identifying control data relating to the application, generating a control score based at least in part on the control data, and assessing the application based at least in part on the risk score and the control score.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 A illustrates a high-level overview of the operation, according to an aspect of the present invention.

FIG. 1 B illustrates a high-level process for entering risk and control information that is being assessed in accordance with an aspect of the present invention.

FIG. 2 illustrates an identifying information page of a risk/control self-assessment tool (SAT) for assessing software applications, according to an aspect of the invention.

FIG. 3 illustrates an index page from the SAT detailing the tabs or links to pages of the tool, in accordance with an aspect of the invention.

FIGS. 4A-4E illustrate risks for which a user may provide input to generate a risk score in accordance with an aspect of the invention.

FIGS. 5A-5H illustrate controls comprising questions/attributes to generate a control score in accordance with an aspect of the invention.

FIG. 6 illustrates a risk and control calculation page, according to an aspect of the invention.

FIG. 7 illustrates a preliminary scoring assessment report, in accordance with an aspect of the invention.

FIG. 8 illustrates an exception report indicating the “no” responses to the questions/attributes provided by a user according to an aspect of the invention.

FIG. 9 illustrates a summary report page from a risk evaluation tool that may be invoked to track applications in accordance with an aspect of the invention.

FIG. 10A illustrates a risk assessment scatter chart and related information, in accordance with an aspect of the invention.

FIG. 10B illustrates the effects of applying specific filters to a risk assessment scatter chart of information in FIG. 10A in accordance with an aspect of the invention.

FIG. 10C illustrates the effects of applying specific filters to a risk assessment scatter chart of information in FIG. 10A in accordance with an aspect of the invention.

FIG. 11A illustrates a risk assessment summary table detailing component control metrics and related information according to an aspect of the invention.

FIG. 11B illustrates some effects of applying specific filters to a risk assessment summary illustrated in FIG. 11A.

FIG. 11C illustrates some effects of applying specific filters to a risk assessment summary illustrated in FIG. 11A.

FIG. 12A illustrates an exceptions summary table detailing statistics about assessed applications receiving “no” in response to a plurality of questions/attributes according to an aspect of the invention.

FIG. 12B illustrates an effect to applying specific filters to the exceptions summary table shown in FIG. 12A, according to an aspect of the invention.

FIG. 13A illustrates an exception detail table that presents identifying information about the applications and individuals responsible for applications receiving “no” responses to the questions/attributes, according to an aspect of the invention.

FIG. 13B illustrates an effect to applying specific filters to the exceptions detail table shown in FIG. 13A, according to an aspect of the invention.

FIGS. 14A-14E illustrates a system of record that may store data about an application, according to an aspect of the invention.

FIG. 15 illustrates a web-based assessment tool that is associated with a system of record, in accordance with an aspect of the invention.

FIG. 16 illustrates an application properties screen of a web-based assessment tool, according to an aspect of the invention.

FIG. 17 illustrates another example of an application properties screen for a web-based assessment tool, in accordance with an aspect of the invention.

FIG. 18 illustrates yet another example of an application properties screen for a web-based assessment tool, according to an aspect of the invention.

FIG. 19 illustrates another example of an application properties screen for a web-based assessment tool, in accordance with an aspect of the invention.

FIG. 20 illustrates an assessment progress report for an application, according to an aspect of the invention.

FIG. 21 illustrates an assessment summary report for an application, in accordance with an aspect of the invention.

FIG. 22 illustrates an example of a scatter chart relating to an application, in accordance with an aspect of the invention.

FIG. 23 illustrates category details for an application, according to an aspect of the invention.

FIG. 24 illustrates a series of questions, responses, and relative scores relating to an application, according to an aspect of the invention.

FIG. 25 illustrates a computing device for implementing an embodiment of the invention.

FIG. 26 is a flowchart that illustrates a method of assessing an aspect of an application.

DETAILED DESCRIPTION

In general, this application relates to assessing an application by identifying risk data, generating a risk score based on the risk data, identifying control data, and generating a control score based on the control data. The application may be assessed based at least in part on the risk score and the control score. Further, this application may embody the method of assessing an application in a computer-readable medium that may include memory and a processor that is configured to execute computer-executable instructions. The computer-executable instructions may include information that may be used to generate a risk score and a control score that is based on risk data and control data, respectively.

The method of assessing the application may be implemented in a variety of embodiments, including the embodiment illustrated in FIGS. 1-13 and another embodiment that is illustrated in FIGS. 14-26. Further, all aspects that are disclosed in each embodiment may be utilized in one or both of the disclosed embodiments of FIGS. 1-13 and 14-26, respectively, and/or any other embodiment that encompasses assessing an application by identifying risk data, generating a risk score based on the risk data, identifying control data, generating a control score based on the control data, and assessing the application based at least in part on the risk score and the control score.

FIG. 1A shows a high-level overview 100 of the operation of an aspect of the present invention, in the context of studying the risks and controls that may be associated with software applications used by an organization. For example, a government agency, a company, a nongovernmental organization (NGO), or other entity may utilize an application during operation.

On the data collection side, a risk/control self-assessment tool 110 (“SAT”) is used by one or more individuals to enter information about each of a plurality of applications 112, 114, and 116. In one embodiment, the manager in charge of that application may be charged with inputting the required information for a corresponding application. Regardless of who enters it, the information is entered into a data store or database 120, associated with a computer.

The database 120 may be a relational database, and may be implemented using a product that may be well known to those skilled in the art of data management and data design.

On the data output side, a risk/control reporting tool 130 (“RET”) may communicate with the data base 120 and may make a wealth of information available that may include the collected data. The reporting tool 130 may be accessed by a wide variety of responsible and/or interested individuals. For example, the people or entities may access an application because they may have a stake in its performance. Such individuals may belong to an organizational hierarchy within a line of business of the organization. As illustrated in FIG. 1, these individuals may include high-level executives 142, technology executives 144, and application managers 146 belonging to a line of business. The high-level executives 142 may have supervisory authority over the technology executives 142. The technology executives 142 who, in turn, may have supervisory authority over the application managers 146. Moreover, there may be no hierarchical structure at all. It may be understood that individuals within a hierarchy may have access to different, and sometimes overlapping, information.

The Risk/Control Self-Assessment Tool (“SAT”) 110 comprises a standard spreadsheet workbook containing multiple tabs. The SAT may be implemented in any desirable application. The SAT tool 110 may permit a submitter to create a record within the database 120 for a new object whose risk and control assessment information is to be added to the database 120. The object may be a software application and the submitter may be the manager responsible for that software application. A variety of other objects and other responsible and/or interested individuals may enter this type of information, as desired. The types of risks that may be faced and the types of controls that may be implemented may vary as appropriate.

FIG. 1B shows a high-level diagram 150 for employing the SAT to create a new record that may be associated with an object. In step 152, a user may enter identifying and parametric information that may uniquely identify the application. The application may include the organization name, the business unit, the application name, costs to date of owning/developing an object, criticality of an object, and information sufficient to identify the people responsible for the object, such as high level managers, project managers, and the like.

In step 154, a user may provide information about various aspects of an inherent risk associated with an object of assessment. Further, information that pertains to predetermined classes that are associated with an object may be assessed. In step 156, the user may provide objective binary information regarding the controls that have been implemented to mitigate the risks that may be organized into one or more control categories. The controls may be organized into a plurality of categories and sub-categories. In step 158, the SAT may calculate a risk score and a control score. In step 160, the user may be able to review the input data before the newly created record is added to the database 120. Of course, the user may edit previously input responses.

FIG. 2 shows a page 200 of the SAT for an embodiment in which the object is a software application. At the bottom of the first page may be a number of tabs that may invoke different modules within the workbook. These may include an index tab 201A, an inherent application risk tab 201B, eight control tabs 201C-201J, and a risk control summary tab 201 K. An assessment report tab 201L and an exceptions report tab 201M that may also be provided. The assessment report tab 201L and an exceptions report tab 201M may provide insight into the information tabulated to date.

The identification screen 200 may request information that may be suitable for initiating a risk assessment. The information may index and retrieve select data for a new application record. The information may be entered into empty cells that are provided. By hovering a cursor over a select cell a pop-up comment 206 may be produced that explains the cell usage. For most screens, clicking in a cell will usually produce a drop-down arrow that will produce a list of eligible answers from which to choose.

A variety of information may be requested which may include first information 202 identifying the interested line of business, second information 203 uniquely identifying the application (i.e., two or more applications may not share the same second information), third information 204A indicating the level of criticality (high, medium, low, and the like) of the application, fourth information 204B indicating the status of the application, and fifth information 204C indicating the total cost of owning/developing the application (TCO).

In addition, sixth information 205 identifying one or more responsible individuals within the relevant business unit may be received. As seen in FIG. 2, this sixth information may include identification of three people within an organizational hierarchy: a high level business unit executive 205A, a technology executive 205B, and the application manager 205C. Thus, information identifying two or more individuals in an organizational hierarchy who are responsible for the application, may be received. Thus, every reporting may reflect three levels of hierarchy that are responsible for the application. In other embodiments, different numbers of people may be specified. Each person may have different titles, and different organizational relationships amongst themselves.

FIG. 3 shows a SAT's tabs index screen 207, which can be reached by clicking on an index tab 201A. This screen 207 may serve as a table of contents for the risk/control self-assessment tool. The tab sub-category 208 and tab description 209 explain the general nature and function of each tab 211. The status 210 of each assessment tab 201A-201K explains whether or not all input has been received for that item. Clicking on a corresponding status link 210A navigates the user to a screen(s) that corresponds to an associated tab.

FIG. 4A shows the first screen 220A of SAT's inherent application risk (JAR) module for a particular application 222 named “Application 1.” The full IAR module contains more questions than are shown on the first screen 215, and these are included in FIGS. 4B-4E on screens 220B-220E, respectively.

The IAR module captures inherent risks to key operational and business attributes of the application being assessed. The risks may be identified within a plurality of risk categories 226. Each risk category may have a set of rules 224 for scoring the risks. A user may provide an answer 228 to each risk category from a drop down menu of choices. Text entry may not be permitted, with an exception for entering the name of the risk assessor. Each of these risk categories 226A, 22613, 226C, and 226D has an associated set 224A, 22413, 224C, and 224D of candidate risk levels. The user may select exactly one risk level from each set after opening a pull-down menu 225 by placing a cursor over the appropriate answer box 228A, 22813, 228C, and 228D.

The candidate risk levels may be ‘ranked’ such that a user electing a higher ordinal number may have the response associated with a candidate risk level for higher risks. For instance, for candidate risk set 22413, option ‘1’ may mean that no other applications depends on the application in question, option ‘4’ may mean that more than 10 other applications depend on this application. The risk increases as the number of dependent application increase. By supplying a risk input (i.e., selecting which option to choose), a user may select a risk level for that particular risk category.

Once all the answer boxes 224 are filled out, the IAR module may calculate a risk score. The risk score may be calculated using the following methodology: (1) not applicable (‘n/a’) responses may be discarded (excluded from all further calculations); (2) the numeric values of all responses may then be summed to create a “numerator;” (3) the maximum possible numeric values of all responses may be summed to create a “denominator;” (4) the numerator may be divided by the denominator to create a “Risk Fraction” (a value between 0 and 1); (5) the Risk Fraction may be multiplied by 10 to create a “Risk Ratio” (a value between 0 and 10); (6) the Risk Ratio may be weighted: it is increased by the percent of total responses that were at their highest possible risk scores (up to a maximum Risk Ratio value of 10); and (7) the Weighted Risk Ratio is used (and shown) on the Risk and Controls Calculator Tab as the “Normalized” value for Inherent Application Risks (JAR).

Other approaches may likewise be employed to arrive at a risk score. A risk score may be substantially commensurate with the relative risks in a population of applications being assessed.

Controls may offset (“balance” or “mitigate”) inherent application risks. FIGS. 5A-5H each present a screen for a particular category of candidate controls. Each category may have many possible controls, but only one screen for each is shown in FIGS. 5A-5H. Each category of controls may be organized into one or more sub-categories, and each sub-category may have one or more attributes.

Attributes may come in different forms. In a common form, an attribute is an affirmative statement regarding the implementation status of a control. An attribute is merely a question that inquires whether a task has been performed and/or completed. Users may be asked to indicate a ‘yes’ or no’ answer to each question. Thus, a user may be expected to give a binary answer that is reflective of the truth status of the attribute. This may be illustrated with reference to a category of controls, as described in detail below.

FIG. 5A presents a first screen 230 for the category of information security controls (ISC) 231 for an application named “Application 1” 222. The ISC category includes a “Governance and Administration” sub-category 232 which is characterized by one or more attributes, designated 233A, 233B, 233C, and the like. The ISC category may include other sub-categories as well. Attribute 233A may be a statement reading “Application security administration processes are documented”. Upon seeing this attribute, the user places a cursor over the corresponding “Answers” box 234A, which may open a menu of possible responses (i.e., “y”, “n,” or “n/a”). The user may select an answer depending on the whether the statement is true or false. The user repeats this process for each attribute within each sub-category for the ISC category. The user may skip an attribute if the user is unsure of the attribute's status. The user may also return to the attribute at a later time, as the SAT may be capable of preserving the responses in the database 120.

FIG. 5B presents a screen 240 for the category of platform/environment controls (PEC) 241 for an application named “Application 1” 222. The PEC tab 201D may capture the effectiveness of the application's platform and environmental controls. The PEC category may include two sub-categories: “Change Control” 242A and “Physical Security” 242B. Each of the sub-categories may have associated attributes, shown generally as 246. The user may respond to the sub-categories 246 with ‘yes’ or ‘no’ inputs.

FIG. 5C represents a first screen 250 for the category of production stability controls (PSC) 251 for the application named “Application 1” 222. The PSC tab captures the status of application controls relating to people, system availability, and problem management. The PSC category may include sub-categories 252A, 252B, 252C, and each sub-category may have one or more attributes. As illustrated in pull-down menu 255, the user may be given a choice of responding with any one of ‘y’, ‘n’ and ‘n/a’ (‘not applicable’) for some attributes. Answer boxes 253 and 254 show that the responses to attributes within a given sub-category may include yes/no responses as well as ‘n/a’ responses when appropriate.

FIG. 5D represents a first screen 260 for the category of vendor management controls (VMC) 261 for an application named “Application 1” 222. The VMC category captures the effectiveness of vendor management controls relating to the application. Because some applications may have no substantial vendor involvement, all answers in this category, except the first, support a ‘n/a’ choice. The VMC category may include six sub-categories 262A, 262B, 262C, 262D, 262E, 262F, each having one or more attributes. In the case of the attributes 266 that are illustrated in FIG. 5D, all responses are binary (of the ‘yes’/‘no’ variety).

FIG. 5E presents a screen 270 for the category of regulatory compliance controls (RCC) 271 for the application named “Application 1” 222. The RCC category may capture the application's adherence to certain regulatory and compliance controls. The RCC category may include five sub-categories 272A, 272B, 272C, 272D, 272E, 272F, each having one or more attributes, shown generally as 276. Because some applications may have no substantial regulatory or compliance involvement, all answers on this tab, except for the first, may support an “n/a” choice.

FIG. 5F represents a screen 280 for the category of technology architecture controls (TAC) 281 for the application named “Application 1” 222. The TAC category measures the viability of the application's technology architecture in addressing growth and business needs. The TAC category may include two sub-categories 282A, 282B, each having one or more attributes, shown generally as 286.

FIG. 5G represents a screen 290 for the category of business process controls (BPC) 291 for an application named “Client Management” 299. The BCC category assesses the effectiveness of the application's business continuity and recovery controls. The BPC category may include sub-categories 292A, 292B, 292C, each having one or more attributes. The BPC tab assesses the application's ability to satisfy select business process needs.

FIG. 5H illustrates a screen 300 for the category of business continuity/disaster recovery controls (BCC) 301 for an application named “Application 1” 222. The BPC category may include at least three sub-categories 302A, 302B, 302C, each having one or more attributes, shown generally as 296. The BCC tab assesses the effectiveness of the application's business continuity and recovery controls.

FIG. 6 shows a screen 320 presenting information about a calculated risk score 322 (designated by variable RO) and a calculated control score 324 (designated by variable CO). The SAT's built-in risk calculator may dynamically compute the assessed application's risk and control scores and may present these results numerically and graphically. Each score may be the result of applying a weighting function to a normalized score. The normalized risk metric 326 (designated by variable NO) is scaled by a weight factor 327 (in this case, 100%) to arrive at the risk score 322. Each member of the plurality of normalized component control metrics 328 (designated by variables N1, N2, . . . , N8) may be scaled by a corresponding weight factor 329 (designated by variables M1, M2, . . . , M8, that totals 100%) to arrive at weighted metrics (designated by variables L1, L2, . . . , L8). The weighted metrics may be summed, to arrive at the control score 324. A value representing the difference, or gap 332 (designated by variable GO=RO−CO) between the risk score 322 and the control score 324 may be displayed. A graphical comparison 2533 of the risk score 322 and the control score 324 may allow the user to view and compare the risk and control scores. In screen 320, one or more of the weights M1, M2, . . . , M8 can be zero. For example, if weights M4 and M5 are zero, this would mean that the application is not a vendor supported and/or that the business activity supported by this application is subject to regulatory scrutiny.

A person having ordinary skill in the art will recognize that the values M 1, M2, M3, M4, M5, M6, M7, and M8 of the individual weight factors 329 will depend on the level of importance for each of the control categories as viewed by the tool's designers, users, and others. A user may be permitted the flexibility to modify the weight factors 329. The sub-categories and attributes may be evaluated for every application that is assessed so that the resulting risk/control metrics can be compared across all applications.

For each control category, the normalized component control metric 328 is calculated as follows: (1) first, not applicable (‘n/a’) responses may be discarded and excluded from all further calculations. If all responses for a given category are ‘n/a’, then that entire category is thrown out and its weight becomes 0%. To maintain a total weighting for all control categories of 100%, all other category weights are adjusted. Only the VMC and RCC categories have the potential to assume 0% weight. (2) The total number of “y” responses may be counted to create a “numerator” and the maximum possible number of “y” responses may be counted to create a “denominator.” (3) The numerator may be divided by the denominator to create the “Control Fraction” (a value between 0 and 1), and the Control Fraction may be multiplied by 10 to create the “Control Ratio” (a value between 0 and 10). The control fraction may be a ratio between the total number of affirmative responses for a particular attribute and the maximum possible number of affirmative responses. (4) The Control Ratio may be adjusted or left unchanged as follows: (a) If the percentage of “no” responses is >60% of total responses, the Control Ratio is reduced by 75%; (b) if the percentage of “no” responses is >40% but less <60% of total responses, the Control Ratio is reduced by 50%; (c) if the percentage of “no” responses is greater >20% but <40% of total responses, the Control Ratio is reduced by 20%; and (d) otherwise, the Control Ratio is left unchanged (at its full value). The adjusted Control Ratio may be used as the normalized component control metric 328 for the corresponding category.

Other approaches, algorithms, and the like, may be used for the purpose of calculating normalized control metrics 328. For example, the control ratio may not be adjusted but instead may be used as the normalized component control metric.

FIG. 7 shows a screen 340 representing an assessment summary report for an application named “Application 1” 222. The assessment summary report may be created before the data is analyzed and consolidated into portfolio views. A first image 342 plots the application's criticality 343 and total cost of ownership 344. A second image 345 maps the application's risk 346 versus its control 347. Summaries 348 of the component risk and control scores are shown at the top of the summary report. These summaries include information 205 identifying the responsible individuals, the application's name 222, the risk score 322, the control score 324, and the normalized component control metrics 328. The SAT's assessment summary report may be reviewed with the application manager during mitigation discussions and planning.

FIG. 8 shows a screen 350 presenting an exceptions report that displays the responses 358 (“yes,” “no,” and “n/a”) to various attributes 352. The “no” responses or any other grouping of the available responses may be displayed. These responses may be displayed in the context of the information 205 identifying the responsible individuals, the application's name 222, the risk score 322 (designated here by variable R2), the category 354, and subcategory 356. Like the SAT's assessment summary report, the exceptions report may be reviewed with the application manager during mitigation discussions and planning.

The foregoing description of the SAT examined specific risk factors, specific control categories, and specific attributes. While these particular factors, categories, and attributes may be suitable for assessing an application, they may not be appropriate for assessing the risks and controls associated with some other type of object. Other risk factors, control categories, and attributes may be developed and used.

Risk/Control Reporting Tool

The Risk/Control Reporting Tool (RET) may be any commercially available workbook containing multiple tabs. The results from the individual SATs that have been aggregated into the database 120 may be loaded into the RET. The RET provides interested personnel with a variety of portfolio views, across all applications that have been entered. For example, personnel such as group executives, technology executives, application managers, risk managers, and others may wish to view the portfolio views. If desired, privileges may be established that permit a user to access a portion of the aggregated data, such as information about those applications that are of interest to the user or that may be assigned to the user by a line of business. The RET may be provided with a collection of filtering and query capabilities having embedded data links to enable interested personnel to quickly identify and focus upon risk/control imbalances (gaps) within assessed applications and across application groups.

FIG. 9 shows the assessment report header summary page 400 of the RET. The summary page 400 presents information about the goals and progress of completing assessments across all applications throughout an organization. For each line of business 402, the summary page may provide information about the number of applications 404 (designated by variables S 11, S 12, . . . , S 16) belonging to a particular division or line of business, the number of applications that have been assessed 406 using the SAT (designated by variables S21, S22, . . . , S26), the percentage of assessed applications 408 (designated by variables S31, S32, . . . , S36), and the goal 410 (designated by variables S41, S42, . . . , S46) for to the number of applications that may be assessed in a particular calendar year or other desirable risk assessment period. Aggregate values for these measures 412A, 412B, 412C and 412D), designated by variables T1, T2, T3 & T4, respectively, may be provided across all the lines of business.

A plurality of tabs may be positioned at the bottom of the summary page. Tab 415A may be for the current view 400; tab 415B may direct the user to a risk assessment scatter plot page 430; tab 415C may direct the user to a risk assessment summary page 450; tab 415D may direct the user to an exceptions summary screen 470; and tab 415E directs the user to an exceptions detail page 490, each of which is described in further detail below.

FIG. 10A shows a “Scatter Chart” page 430 comprising a scatter chart 432 that may be a multi-dimensional graphical plot for all assessed applications. The scatter chart 432 plots risk scores 434 having a range of 0-10 on the X-axis plotted against control scores 436 having a range of 0-10 on the Y-axis. The scatter plot may focus upon those applications for which the risk/control balance may not be acceptable. Various indicia may provide an objective measure of a combination of criticality and/or cost of a particular data point and may allow a viewer to focus on important data points. The indicia may be icons of different shapes, such as the shape of solid circles 437A, solid triangles 437B, solid squares 437C, and the like. The various icon designs and plotting patterns may be plotted above colored backgrounds to denote a relative presence or absence of commensurate risk and control. For instance, a red background may denote areas of the scatter plot in which the level of control is likely unacceptable in view of the risks, a green background may denote areas of the scatter plot in which the level of control likely is acceptable in view of the level of risk, and a yellow background may denote areas of the scatter plot in which it is not clear whether the level of control is acceptable in view of the level of risks.

As illustrated in FIG. 10A, applications plotted into sectors having a combination of high risk scores and low control scores, such as sectors 441A, 441B, 441C, and 441D, may be considered under-controlled and may deserve prompt attention. Applications having both high control and high risk, such as sector 442A, may be considered to have an adequate risk/control balance. Applications having low-to-moderate risk and high control, such as in sector 443A, may be considered potentially over-controlled and subject to an evaluation to determine whether some of the controls may be relaxed and costs may be saved. Some of these applications may be designated by the line of business as a mandatory application. Applications may be categorized into one of the above groups, or even newer groups, depending on the risk/control preferences of the organization.

The entire scatter plot 432 may be divided into either larger or smaller sectors depending on the position of the grid lines. Portions of the scatter plot 432, such as individual sectors, may be color-coded. For example, ‘red’ may indicate under-controlled, ‘green’ may indicate adequately controlled, and ‘yellow’ may indicate over-controlled.

The scatter plot 432 may be accompanied by columns of information about all of the plotted applications. The columns of information may include an identification of the individuals that are responsible 205 for each assessed application, including a group executive 205A, a technology executive 205B, and the application manager 205C. The columns may also include the application name 222, the risk score 322 for that application, the control score 324 for that application, and an indication 446 of the criticality/cost of the application.

When pressed, drop-down arrows 448 in the column headings may present filtering selections for the related topics. Filters can also be used to narrow focus on items of interest. The scatter plot and all related data may dynamically change to display only those items to which the filters apply.

For instance, each of the columns for the responsible individuals 205A, 205B, and 205C may be filtered so that the scatter plot will show only applications for which a subset of all individuals are responsible. Boolean operators may be used within a single column to plot applications for which a particular desired combination of individuals is responsible. For example, one may filter only the “Group Executive” column 205A to request that the scatter plot 432 display applications for which two specific group executives are responsible. Filtering multiple columns may perform a logical “AND” operation. The scatter plot will show applications at the intersection of the responsible individuals from each filtered column.

In addition, one may filter the risk scores 322 to specify a range of risk scores, or even non-contiguous ranges of risk scores using Boolean operators. The control scores 324 may be filtered in the same manner. Filtering may also be used to identify the applications that meet a predetermined criticality/cost criterion 446 for the applications that are present in a particular sector.

Each assessed application may have a corresponding data point in some multi-dimensional hyperspace that may include three users, a risk score, a control score, a criticality parameter, cost parameters, and the like, as illustrated in FIG. 10A. One may use the RET to focus on a particular subspace of the multi-dimensional hyperspace or on a single application by using the provided filtering capabilities. The resulting filtered scatter chart thus enables views of individual applications or application groups along dimensions of inherent risks, controls, business criticality and application, and total cost of ownership.

FIG. 10B shows a filtered scatter plot 450 resulting from specifying (i.e., filtering) the following responsible individuals: group executive (named “Person 4”) 451, technology executive (named “Person 6”) 452, and application manager (named “Person” 17) 453. The applications having this combination of responsible individuals may be plotted on the scatter plot 450, and the other ‘filtered’ columns 454 present the corresponding values and parameters. The RET can be used as an accountability tool that links responsible individuals to the applications for which they are responsible.

FIG. 10C shows a filtered scatter plot 460 resulting from identifying the applications that have a risk score 462 greater than 8.0 and a control score 464 less than 7.0.

A “reset” button 479 can be used to restore the view so that all filters are set to 100% (i.e., no filtering) and the assessed applications are included on the scatter plot once again. This permits a viewer to continue to study the data with a different combination of filters.

FIG. 11A shows risk assessment summary page 470 that displays a table 472 showing eight normalized component control metrics 474 and the control score 324 for the assessed applications. Each control category may correspond with one of the eight normalized component control metrics. This page 470 allows a viewer to study the individual control components which resulted in the control score.

Individual metrics may be colored in different ways, depending on their values. For instance, low-valued metrics may be represented in red 478R, high-valued metrics may be displayed in green 478G, intermediate-valued metrics may be displayed in yellow 478Y, and white spaces 478W may be used for those instances in which that particular control category was not applicable. Filtering can be performed on any combination of the columns, including the risk score 322, the responsible individuals 205A, 20513, 205C, and the application name 222.

FIG. 11 B shows a filtered risk assessment summary page, in which the table 480 results from identifying (i.e., filtering) the following responsible individuals: group executive (named “Person 4”) 451, technology executive (named “Person 6”) 452, and application manager (named “Person 17”) 453. The risk assessment summary may be displayed for the applications having this combination of responsible individuals to provide the individuals with insight on the efficacy of the combination of implemented controls that are being used to combat risks.

FIG. 11 C shows a filtered risk assessment summary page, in which the table 490 results from identifying the applications having a risk score 462 greater than 8.0 and a control score 464 less than 7.0.

FIG. 12A shows an exceptions summary page 500 that represents control exceptions 502 for each application within the assessed portfolio. In this context, a ‘control exception’ may be a question/attribute to which the answer was “no” or a “no” response having designated the absence of a desirable control. In its initial display, summary page 500 may rank the questions/attributes in descending order, based on the percentage of “no” responses obtained by SAT. A number of columns containing pertinent information may be displayed on this page 500. These include an indication of the control category 504, the control subcategory 506, the text of the attribute 508 to which the “no” response was given, a question key index 510, the total number of “no” responses 512 (the numbers being represented by X1, X2, and the like), and the percentage of no responses 514 (the percentages being represented by Y1, Y2, and the like). Each of these columns may be filterable.

FIG. 12B shows a filtered exception summary page that displays the results from identifying (i.e., filtering) the category 504 and sub-category 506 columns of the exception summary page 500. In FIG. 12B, the category filter was set to “Info Security Controls” 516 and the sub-category filter was set to “Governance and Admin” 518. Seven questions/attributes had “no” responses (the numbers being represented by V 1, V2, and the like) to this category/sub-category combination across all of the assessed applications. The “no” responses are displayed, in descending order of percentage of “no” responses (the percentages being represented by W I %, W2%, and the like). A viewer may set additional filter parameters, such as ‘only showing “no” percentages greater than 15%’, and the like.

FIG. 13A shows an exceptions detail page 530. This page 530 enables viewing of specific control weaknesses at any organizational level within the assessed application population. The columns on this page include the responsible individuals 205A, 205B, 205C, the application name 222, the risk score 322 (designated here by variable R1), the control category 504, the control sub-category 506, the text of the question/attribute 508, and the unique question key index 510. Each of these columns may be filtered. Counts 532 reflective of the population of some of the columns may be generated to provide on insight into the exceptions.

FIG. 13B shows a filtered exceptions detail page which results from specifying (i.e., filtering) the following responsible individuals: group executive (named “Person 4”) 451, technology executive (named “Person 6”) 452, and application manager (named “Person 17”) 453. Only the applications having this combination of responsible individuals and for which a “no” response was assigned to a question/attribute 508, are shown. The RET allows a viewer to study the data that is collected from the SAT from various perspectives.

In one respect, the SAT is an application that is configured to collect information about risks and controls implemented to mitigate the risks. The risks and controls may be associated with a plurality of applications. The SAT is configured to: (1) receive information that uniquely identifies each application, (2) receive risk input that is reflective of a risk level associated with each of a plurality of risk categories, (3) compute at least one risk score based on the risk input, and (4) receive a plurality of control responses indicative of whether or not each of a plurality of control attributes has been implemented.

The RET is an application that may be configured to identify the nature of the risks and the controls that may be implemented to mitigate the risks. The risks and controls may be associated with a plurality of applications. At a basic level, the RET may be configured to display information that is associated with both calculated risk scores and calculated control scores for a plurality of applications and may permit filtering to study only a portion of the information.

The SAT and the RET are intended to be employed by the same organization, but may also be separately implemented. Thus, in some embodiments, the SAT and the RET may be combined into a single Applications Risk and Control Assessment Tool (ARCAT). However, when appropriate, they may be separated, either as distinct applications or through privilege settings. A user may have access to one or both of the SAT and the RET.

The risk and control assessment tool relates to the assessment of risk applications and control applications. Each risk and control application may be implemented in one or more lines of business. Applications may be developed to help a line of business to execute its assigned tasks. The applications may provide a line of business with the ability to analyze data, store data, organize data, and generally create efficient methods and systems of assessing the application to streamline the processes for which the line of business may be responsible.

FIGS. 14-26 illustrate another example of aspects of the invention. Many lines of business wish to assess an application to determine whether the risks and controls are generally commensurate with one another. A report may be generated to reflect the data that is collected during the application's assessment. Various levels of reporting hierarchy may be established, depending on the type of application, position of the user of the application, and/or any other desired feature.

Methods of assessing an application or group of applications may be web-based. For example, a user may have access to software for assessing an application or group of applications over a webpage on an intranet. For example, a line of business may operate an intranet and may permit users to access the software by connecting to an Internet Protocol (IP) address on a private network. The software may require that the user input a username and password before permitting the user to access the application. The software may also require that the computing device from which the user accesses the software may contain an identification verifier such as a digital certificate that confirms the user's identity.

Further, the assessment software may integrate with another software program or group of programs. The integration between software applications may provide a user with expedited analysis on data and prevents a user from repetitively inputting the same data into multiple programs. Additionally, the integration feature between software programs automates the process of analyzing existing software applications and reducing redundant risk assessment processes.

A variety of users may be permitted to access the assessment application. For example, an application manager, risk manager, chief-information-officer (CIO), and a technical executive may each be permitted to access the assessment application. An application manager may be responsible for creating, maintaining, and reviewing application information for each application within a group of applications and/or performing the assessment of each of the applications. A risk manager may be responsible for reviewing assessment results for an application and identifying any issues that need to be mitigated or to which the line of business may need to attend. A CIO and/or technical executive may be responsible for understanding a line of business' needs and goals and may select the risks and controls that are appropriate for a line of business. Further, the CIO and/or technical executive may be permitted to prioritize the risks and controls and decide whether mitigation efforts are needed.

A group of users may be assigned a priority level. Further, a first user may be assigned a first priority level and a second user may be assigned a second priority level that is different from the first priority level. The priority level that is assigned to a user may correspond to the user's employment position with the line of business. For example, an application manager may be permitted to create, maintain, and review information relating to an application whereas a CIO may be permitted to make decisions regarding the types of risks and controls that are acceptable to the line of business and that perform analysis upon the information that is created, maintained, and reviewed by the application manager.

A system of record may be an Application Inventory Tool (AIT) 1402. The AIT 1402 may store data about one or more applications that may be associated with a line or lines of business. For example, the AIT 1402 may serve as the system of record for risk and control data fields. The AIT 1402 may store a profile containing information relating to the applications associated with a line of business.

FIG. 14A illustrates a search screen associated with the AIT 1402. The search screen may contain one or more search criteria such as an application identifier (ID) 1404, an application name 1406, an application short name 1408, an application description 1410, a supporting company or cost center 1412, a supporting hierarchy code 1414, a management contact 1416, technological support or development contact 1418, and an application status 1420.

The application ID 1404 may contain information relating to the identifier of an application such as an ID number assigned to the application. The application name 1406 may be the name assigned to the application by the line of business or any other entity. The application short name 1408 may be the short name, nick name, internal name, or the like that is assigned to an application by a line of business or any other entity. The application description 1410 may include a description of the application and may include information relating to relevant lines of business, statistics, or other desired information.

The support company or cost center 1412 may include information relating to the line of business that funds the application's operation. Further, the support company or cost center 1412 may also include information relating to the entity to which the application's financial information may be sent. The supporting hierarchy code 1414 may include information relating to the designation of functional business entities or business units. The hierarchy code 1414 may include a range of codes from 1 to 10 characters that indicate the corresponding functional entity. The management contact 1416 may include information relating to the individual, group of individuals, line or lines of business, and/or entities that are responsible for the management of the application. The AIT 1402 may contain any search criteria, as needed.

A user may input information into one or more of the search criteria and select a search feature 1422. FIG. 14B illustrates a specific example of a search for the application entitled “Application Inventory Tool (AIT)” 1402. The user may input information into each of the search criteria such as the application identifier (ID) 1404, the application name 1406, the application short name 1408, the application description 1410, the supporting company or cost center 1412, the supporting hierarchy code 1414, the management contact 1416, the technological support or development contact 1418, and the application status 1420. In this example, the user may input information relating to each of the search criteria. The search may be performed when a user inputs information into at least one search criteria and does not require that information is input into more than one search criteria.

FIG. 14C illustrates a search screen that may be associated with an AIT 1402 and includes additional search criteria including a requested application status 1422, a legacy organization 1424, an application support and usage 1426, an internal or external customer criteria 1428, a geographical location of users criteria 1430, a substantial vendor/supplier relationship name 1432, and an inherent risk index (IRI) 1434. The request application status 1422 may include information relating to the status of the application, e.g., whether it is in development, in production, on hold, in action, scheduled to be retired, retain data and information about the application, retain documentation about the application's data only, whether decommissioned, and the like. The legacy organization 1424 may include information relating to information and data about a previous organization's use of an application, a previous line of business' experience with the application, and the like.

The application support and usage 1426 may contain information relating to the technology infrastructure of the line of business. For example, the application support and usage 1426 includes information about special or unique features of the application that may differ from the standard features used in an application. The internal or external customer 1428 may indicate whether the application is designed to serve a line of business within the entity or at a third party (e.g., whether the application is internal and/or external). Additionally, the geographic location of users 1430 may include information relating to the city, state, region, country, time zone, or the like in which the user is likely to be located.

The substantial vendor/supplier relationship name may indicate information and/or data about the vendor who developed, maintains, and/or supports the application code. Further, the IRI 1434 may include information relating to the inherent risk indicator for a supply chain and may include assigning a rating to the application.

One or more of the search criteria may contain an option to select “N/A” 1436 to indicate that the selected search criteria is “not applicable” to the application. Further, one or more of the search criteria may be required to operate ARCAT and may be indicated by an ARCAT symbol 1438. Such a symbol may indicate to the user that the search criteria may be required for ARCAT or may be important to the generation of a risk, control score, and/or any other aspect of assessing an application.

In the example illustrated in FIG. 14C, the ARCAT symbol 1438 indicates that the search criteria with which the symbol is associated is required for a risk analysis and calculation of a risk score.

Further, a link may be provided that provides the user with ARCAT Control Questions 1440. The ARCAT Control Questions 1440 may include seven categories, including but not limited to: technology architecture, platform/environmental, production stability, vendor management, regulatory compliance, business process, and information security. The link may also present the user with one or more questions, as illustrated in FIGS. 14D and 14E. For example, the ARCAT Control Questions 1440 may be divided into several categories, including, but not limited to, capacity 1442, scalability 1444, change control 1446, and physical security 1448. In the examples illustrated in FIGS. 14D and 14E, the user may select either “yes” or “no” in response to each of the ARCAT Control Questions.

The Capacity 1442 category may include questions such as: whether the platform upon which the application resides has adequate capacity for growth to meet the current and anticipated future business needs; whether the application has adequate capacity for growth to meet current and anticipated future business needs; and whether the network components utilized by the application have adequate capacity for growth to meet the current and anticipated future business needs. The Scalability 1444 category may include questions such as: whether the application, its database, and supporting platform will scale to accommodate anticipated future business needs.

The Change Control category 1446 may include questions such as: whether a formal bank-approved and documented change control process is used to manage all changes to the application; whether programmers are denied the ability to create, modify, or delete production code and/or production data, unless specifically authorized (for every occurrence); whether development, test, and production environments are segregated and access is restricted in accordance with job responsibilities; and whether a formal documented emergency change control process is in place and whether the formal documented emergency change control process is followed.

In some embodiments, a web-based assessment tool may be associated with the AIT, such as ARCAT, as described in detail above. As illustrated in FIG. 15, an applications screen 1502 may be accessible by the user, such as a webpage on an intranet that may be associated with a line of business. The applications screen 1502 may include an application filter 1504 that may provide criteria on which ARCAT may search for an application, similar to the search criteria described above for the AIT in FIGS. 14A-14E. The search conducted in ARCAT may include such search criteria as an application's name 1506, an AIT #1508, an AIT short name 1510, an OBS unit 1512, an OBS filter mode 1514, an application manager 1516, a status 1518, a criticality 1520, a Sarbanes-Oxley (SOX) application flag 1522, and a data load valid flag 1524.

The application name 1506, the AIT #1508, and the AIT short name 1510 may include identification information relating to the application. Further, the application manager 1516 may include information relating to the individual, individuals, line of business, and/or entity that may be responsible for managing the application. The status 1518 may include information relating to the application's status.

The OBS Unit 1512 may include information relating to bank hierarchies. Further, the OBS Filter Mode 1514 may include information and also relates to bank hierarchies. The OBS Unit 1512 and the OBS Filter Mode 1514 may not be included as a portion of the application risk and control assessment tool.

The criticality 1520 may include information relating to the level of criticality of the application and may include information relating to the business impact associated with an assessment of the application. The SOX application flag may indicate whether the application contains information that may be relevant to the SOX reporting that may be required by the federal government for financial institutions. The SOX application flag may be used for both risk scoring and for triggering SOX risk assessments. A SOX risk assessment may be triggered by any desired event or expiration of a time period. A SOX risk assessment may also be triggered automatically and/or manually. The data load valid flag 1524 indicates whether the data that is associated with an application is confirmed to be valid data.

A user may select to filter a collection of applications by the search criteria input into the application filter 1504. For example, the user may input an application name 1506, a status 1518, and a SOX application flag 1522. Next, the user may select to filter the applications by the selecting a filter button to commence the filtering processes by the search criteria: name 1506, status 1518, and SOX application flag 1522.

The applications screen 1502 of FIG. 15 may also include search results 1518. The search results 1518 may be organized by application name 1506 or any other desired search criteria. In FIG. 15, the search results may be sorted or selected based upon any available search criteria.

Further, one of the applications in the search results is entitled “Banking Center Online” 1528. The “Banking Center Online” 1528 application may include a status of “in production” 1530 to indicate that the application is the process of being created and/or tested. The “Banking Center Online” 1528 application may include an “M” 1532 in the criticality column that may indicate that the application is of a medium criticality to the line of business. Further, the “Banking Center Online” 1528 may include an “X” 1534 in the data load valid flag column, as illustrated in FIG. 15. The “X” may indicate that the data that is associated with the “Banking Center Online” 1528 application may not be valid or may not be complete.

In another example, an application may appear in the search results, such as “Channel Technology IU” 1536 that may include a check mark 1538 in the data load valid flag column. The check mark may indicate that the data that is associated with the “Channel Technology IU” 1536 application may be complete or may be confirmed to be valid.

FIG. 16 illustrates an application properties screen 1602. The application properties screen 1602 may include a plurality of menus including, but not limited to, a general application properties 1604, an assessment details 1606, an inherent application risks (IAR) 1608, a business continuity/disaster recovery controls (BCC) 1620, a technology architecture controls (TAC) 1622, a platform/environmental controls (PEC) 1624, a production stability controls (PSC) 1626, a regulatory compliance controls (RCC) 1628, a business process controls (BPC) 1630, and an information security controls (ISC) 1632.

The application properties screen 1602 that is illustrated in FIG. 16 is displaying the general application properties 1604 for an application entitled “Channel Technology UI.” The general application properties 1604 may include general information such as the application's ID, the application's name, the date the application was created, and the date the application was last updated. The general application properties 1604 may also include the application's profile that may include such information as the AIT short name, the AIT #, the hierarchy code, the criticality, the application TCO, the production date, the status, the data load valid flag, the SOX application flag, the VMC exempt flag and the RCC exempt flag the organizational breakdown structures including such information as the hierarchical code. The VMC exempt flag may be set when no vendor is involved in a particular application. The RCC exempt flag may be set when no regulatory oversight is involved with the business activities that are associated with a particular application.

The application properties screen 1702 that is illustrated in FIG. 17 is displaying the assessment details 1704 for an application entitled “Merlin—Channel Technology Mid Tier.” The assessment details may include a plurality of categories including an assessment summary 1706 and assessment details 1708. The assessment summary 1706 may include information such as a risk score, a control score, a gap, and a date of the last assessment of the application. The assessment details may include an IAR score, a BCC score, a TAC score, a PEC score, a PSC score, a VMC score, an RCC score, a BPC score, an ISC score, and a SOX score. Each of the scores that are associated with a risk score and/or a control score may be individually scored to help determine areas of relative strength and weaknesses that are associated with an application. For example, a line of business may assess an application for its ability to mitigate risks and may focus its assessment of the application on individual control scores.

The application properties screen 1802 that is illustrated in FIG. 18 is displaying the Inherent Application Risks (JAR) 1804 for an application entitled, “Channel Technology IU.” The IARs may include, but are not limited to, information relating to whether the users will be internal or external to the line of business, the level of criticality assigned to the Business Impact Assessment (BIA) (e.g., low, medium, or high), the total cost of ownership (TCOA), the IRI, the upstream application dependencies, the downstream application dependencies, the frequency of product code changes (e.g., how often the production code is updated), the peak number of concurrent users, the response time requirement for the application, the peak transaction rate experienced by the application, the user access pattern (e.g., how frequently a user may access the application and/or the length of time for which a user may access an application), the percentage of data periodically reviewed and certified as accurate, and the average number of hours per week that a user may work on the application.

In FIG. 19, an application properties screen 1902 is illustrated that displays the Business Continuity/Disaster Recovery Controls (BCC) 1904 for an application entitled “Channel Technology UI.” The BCCs 1904 may include, but are not limited to, such information as the business continuity planning and maintenance that is related to an application 1906, the business continuity testing 1908, and the like.

FIG. 19 also illustrates several additional categories such as the technology architecture controls (TAC) 1910, the platform/environmental controls (PEC) 1912, the production stability controls (PSC) 1914, the regulatory compliance controls (BPC) 1918, and the information security controls (ISC) 1920. Each of the TAC 1910, the PEC 1912, the PSC 1914, the BPC 1918, and the ISC 1920 may have individual control criteria that may be assigned a significance including most significant, moderately significant, and low significance.

The TAC 1910 may include information relating to whether the platform on which the application resides, the network components, and the application itself have adequate capacity for the growth that may be experienced by the line of business. Further, the TAC 1910 may include information relating to the scalability of the application, its database, and its supporting platform and whether it can meet the future business needs of the line of business operating the application.

The PEC 1912 may include information relating to the manner in which a change may be made to the application. The PEC 1912 may include information relating to whether an approval process is required for changes to be made to the application. For example, the line of business may require approval of each change or may provide general approval to a user based on the user's level of privilege. Further, the changes may permit or deny a programmer the ability to created, modify, or delete production code and production data unless they are specifically authorized for each occurrence via an approved change control process.

The PEC 1912 may also include information relating to the development, testing, and production environments, whether these environments may be segregated, and whether user access to the environments may be restricted. The PEC 1912 may also include information relating to the formal documented emergency change control process and whether it is obeyed during a specific action.

The PEC 1912 may also include information relating to the physical security of the application that includes information on the mainframe, the server, the computing devices, and the like upon which the application resides. For example, the PEC 1912 may include information on whether the will be stored in a Tier 1 facility. Further, the PEC 1912 may include information about whether a formal documented process exists for granting and maintaining access to the physically secure environment and information about the physical access controls and authorization process. Information regarding the review of the documents and/or controls and processes may be included.

The PEC 1912 may also include information about the contractors, visitors, and/or other access of the application by non-authorized users. Information about all non-authorized users' access to an application may be included in the PEC 1912.

The PEC 1912 may include information about the environmental controls related to the application including, but not limited to, the continuity of electrical power and its periodic testing results and whether the physical components of the application have adequate protection and containment of physical damage and threats such as fire, water damage, heating, cooling, and humidity controls, and the like. The PEC 1912 may include information about the production processing for the application and may include current and previous versions of application code and related data that may be backed up and stored offsite from the application and offsite versions of code and data that may be periodically tested to ensure readability and recoverability.

The PSC 1914 may include information about the application's primary and backup associates and/or contractors related to the line of business that operates the application. Further, the PSC 1914 may include information about the training and reference materials for the users of the application and may retain information about the application's maintenance and history of maintenance. Additionally, the PSC 1914 may include information about the system availability that relates to whether the application operates at its expected performance levels and whether its documented service level is acceptable to the line of business. The service level may indicate increases and/or decreases in the maintenance required to operate the applications and whether those services are being timely performed. The PSC 1916 may also include information about problem management and may relate to documentation of problem communication and escalation procedures that have been established for an application (e.g., whether problems are handled in a timely and effective manner).

The RCC 1916 may include information about the regulatory requirements for the application's development specifications. The application itself may include information about its compliance with regulations before it is in operation and may be periodically monitored for compliance with regulations and any other standards requirements. Reports on the application's compliance with regulations may be generated and may identify actionable service, improvement, and/or problems. The RCC 1916 may also provide information about international implications for the application and relocation options and information for the application including whether the application is compliant with a particular country's regulations, whether the necessary infrastructure exists for a particular application, whether a particular country may provide adequate security for the application, and the like.

The BPC 1918 may include information about audit issues, reporting and reconcilement, and data processing of the application. The BPC 1918 may include financial and non-financial information about all of the reports that an application may offer a user and information about whether the data that is included in a report has been verified and/or authorized by specific users. The data processing may include detection of erroneous or invalid data and transaction entries. The errors may be documented, reconciled, and/or included in an audit of the application. The audit may include information about the errors including, but not limited to, who or what caused the errors, the time and place at which the errors occurred, and the like.

The ISC 1920 may include information about whether the application logs all transaction activity by a user and whether the user produces reports that are viewed, saved, printed, and the like. The application logs may also include information about the frequency of the production of reports and whether the application's user authentication method is properly functioning. The ISC 1920 may additionally include information about the data security of the application. The data security may include confidential, proprietary, and/or public information. The ISC 1920 may include information on whether the application functions in accordance with the information that it contains (e.g., treating confidential and proprietary information with high levels of security).

Each of the TAC 1910, the PEC 1912, the PSC 1914, the BPC 1918, and the ISC 1920 may include any desired information and may be periodically reviewed and updated to meet the needs of the line of business, government regulations, or any other compliance standard.

As illustrated in FIGS. 20-22, ARCAT may provide information relating to the application or applications that are used by a line of business. For example, a line of business named “FF-Consumer-Small Business Technology Group” 2004 may be interested in obtaining information relating to all or a portion of the applications it uses. The FF-Consumer-Small Business Technology Group 2004 may obtain information relating to an assessment progress 2008, an assessment summary 2010, a scatter chart 2012, an exception summary 2014, and a data load exceptions 2016.

FIG. 20 illustrates an assessment progress report 2008 for the FF-Consumer-Small Business Technology Group 2004. The assessment progress report 2008 may include information on twenty-three applications 2006 that may be utilized by the FF-Consumer-Small Business Technology Group 2004. Further, it may include information such as the total number of assessable applications, the total assessed applications, the total % of applications that were assessed, the total mandatory assessable applications, the total mandatory assessed applications, the total optional assessable applications, the total optional assessed applications, the total optional % of assessed applications, and the like.

FIG. 21 illustrates an example of an assessment summary report 2104 for the FF-Consumer-Small Business Technology Group 2102. The assessment summary report 2104 may include individual control scores by category as well as overall risk and overall control scores.

FIG. 22 illustrates an example of a scatter chart displaying data from an assessment of an application.

FIGS. 23-26 illustrate the category details for an application. FIG. 23 illustrates a category-wise scores filter 2302 that contains information relating to a plurality of question categories including IARs, BCCs, TACs, PECs, PSCs, VMCs, RCCs, BPCs, ISCs, and the like. The category-wise scores filter 2302 may include results for each of the question categories. For example, each of the question categories may have information relating to a link to viewing the category response 2304, an indication of whether the application's data has been certified 2306, a raw category score 2308, an adjusted raw category score 2310, a category weight 2312, and a category weighted score 2314. Any desired algorithm may be used to calculate the category raw score 2308, the category adjusted raw score 2310, the category weight 2312, and the category weighted score 2314. Further, any desired method of analysis may be implemented to create the category response 2304 and to certify the data 2306.

FIG. 24 illustrates an example of a series of questions 2404, responses 2406, and relative scores 2408. The questions 2404 may comprise the text of the questions that may be associated with a particular question category. The responses 2406 may include the text of the responses to the questions 2404. The relative score 2408 may include a comparison of the risk score and the control score or any other scoring analysis that may be desired by the line of business.

Each application may be assessed based on one or more factors relating to the application's risk and control. The risk assessment may include one or more risk factors and the control assessment may include one or more control factors. Portions of both the risk assessment and the control assessment may be performed in a computer system. Moreover, the risk and control assessment may be generated in a computer system.

FIG. 25 illustrates an example of a computing system environment 2500 that may be used according to one or more embodiments of the invention. The computing system environment 2500 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. The computing system environment 2500 should not be interpreted as having any dependency or requirement relating to any one or combination of the illustrated components.

The invention is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

With reference to FIG. 33, the computing system environment 2500 may include a computer 2501 having a processor 2503 for controlling overall operation of the computer 2501 and its associated components, including RAM 2505, ROM 2507, an input/output module 2509, and a memory 2515. The computer 2501 typically includes a variety of computer readable media. The computer readable media may be any available media that may be accessed by the computer 2501 and include both volatile and nonvolatile media and removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media.

Computer storage media may include volatile and nonvolatile and removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), electronically erasable programmable read only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, and any other medium that can be used to store the desired information and that can be accessed by the computer 2501.

Communication media may embody computer readable instructions, data structures, program modules, and/or other data in a modulated data signal such as a carrier wave or other transport mechanism. It may also include any information delivery media. Modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media. Although not shown, RAM 2505 may include one or more are applications representing the application data stored in RAM 2505 while the computer is on and corresponding software applications (e.g., software tasks) are being executed.

The input/output module 2509 may include a microphone, keypad, touch screen, and/or stylus through which a user of the computer 2501 may provide input. The input/output module 2509 may also include one or more of a speaker for providing audio output and a video display device for providing textual, audiovisual, and/or graphical output.

Software may be stored within memory 2515 and/or storage and may provide instructions to the processor 2503 for enabling the computer 2501 to perform various functions. For example, the memory 2515 may store software used by the computer 2501, such as an operating system 2517, application programs 2519, and an associated data file 2521. Alternatively, some or all of the computer executable instructions for the computer 2501 may be embodied in hardware or firmware (not shown). As described in detail below, the data file 2521 may provide centralized storage of the risk and control assessment.

The computer 2501 may operate in a networked environment supporting connections to one or more remote computers, such as computing devices 2541 and 2551. The computing devices 2541 and 2551 may be personal computers or servers that include many or all of the elements described above relative to the computer 2501. The network connections depicted in FIG. 1 include a local area network (LAN) 2525 and a wide area network (WAN) 2529 and may also include other networks. The computer 2501 is connected to the LAN 2525 through a network interface or adapter 2523 when used in a LAN networking environment. The server 2501 may include a modem 2527 or other means for establishing communications over the WAN 2529 when used in a WAN networking environment. For example, computer 2501 may connect to the WAN 2529 such as the Internet 2531 through a modem connection. One of ordinary skill in the art will appreciate that the network connections may include any communications link between computers.

The existence of any of various well-known protocols such as TCP/IP, Ethernet, FTP, HTTP and the like is presumed, and the system can be operated in a client-server configuration to permit a user to retrieve web pages from a web-based server. Any of various conventional web browsers can be used to display and manipulate data on web pages.

Additionally, an application program 2519 used by the computer 2501 according to an embodiment of the invention may include computer executable instructions for invoking user functionality related to communication, such as email, short message service (SMS), and voice input and speech recognition applications.

The computing devices 2541 or 2551 may also be mobile terminals including various other components, such as a battery, speaker, and antennas (not shown). The input/output module 2509 may include a user interface including such physical components as a voice interface, one or more arrow keys, joystick, data glove, mouse, roller ball, touch screen, or the like.

Each of the plurality of computing devices 2541, 2551 may contain software for creating a data file 2521. The software may be a set of detailed computer-executable instructions for the computing devices 2541, 2551. The software provides the computing devices 2541, 2551 with the ability to create a data file 2521. The data file 2521 may contain multiple individual files of information that may each correspond to an individual document. For example, if a plurality of applications are being assessed, then each application's assessment may be separately contained within the data file 2521. Additionally, a report may be generated that includes information relating to one or more applications in the data file 2521. A report may include any desirable information. The report may include information for reporting requirements of financial institutions such as for stock holders and government laws and regulations. The report may include all or a portion of the data in the data file 2521 and may be customized depending on the needs of the line of business generating the report.

The server 2501 may include memory 2515 for storing computer-readable instructions and a processor 2503 for executing the computer-executable instructions. The computer-executable instructions may be data in the form of program source code that is capable of modifying the data file 2521. The computer-executable instructions may be a series or sequence of instructions for a computing device that is typically in the form of a programming language such as C++, Java, SQL, or the like. A person of ordinary skill in the art will appreciate that various computer programming languages may be used to create the computer-executable instructions, and the invention is not limited to the numerous programming languages listed above.

The memory 2515 may be a portion of the server 2501 that stores data or other instructions. The memory 2515 may be retained or lost when power is lost to the system. The memory 2515 may provide access to data for a user or computing device 2541, 2551 to revise and manage a data file 2521 or may only provide access to the data file 2521. These and other aspects of the memory 2515 will be apparent to one of ordinary skill in the art in view of the below description.

The processor 2503 may be capable of executing the computer-executable instructions. The computer-executable instructions may be executed by the processor 2503 after they have been stored in the memory 2515. The processor 2503 may be a centralized element within a computing system that is capable of performing computations. For example, the processor 2503 may perform the computations that are described in the computer-executable instructions and then execute the computer-executable instructions. In accordance with at least one aspect, the computer-executable instructions may include data describing changes to the data file 2521 that were made by a user or computing device 2541, 2551 over the computer network 2531. The server 2501 stores the data in the data file 2521 that may be associated with a customer's account. The data file 2521 may be stored in the memory 2515 so that it may be accessible to a plurality of computing devices 2541, 2551 and/or users.

The data that is stored in the data file 2521 may include risk assessment information and control assessment information. The risk and control assessment information data may be stored in the data file 2521. Security precautions may be implemented to prevent unauthorized access to the data file 2521. A log-on identification and a password may be required to access the data file 2521 and/or the assessment data. Some of the data that is stored in the data file 2521 may be shared between multiple lines of business. One of skill in the art will appreciate that any desirable security precautions may be implemented.

The computer-executable instructions may be a series or sequence of instructions for a computing device 2541, 2551, described in detail throughout this disclosure. The processor 2503 may be configured to execute the computer-executable instructions that may be used to assess an application during a risk or control assessment or another desirable assessment. Such computer-executable instructions may be located (e.g., physically or logically) in modules in the memory 2515. The computer network 2531 may be any network that interconnects users and/or computing devices 2541, 2551. According to at least one aspect of the invention, the computer network 2531 may provide shared access by two computing devices to at least a portion of the data in the plurality of modules. Shared access may be two or more computing devices 2541, 2551 that may be coupled to the computer network 2531 and/or that may be able to communicate with each other and/or access, change, and add data to a data file 2521.

The computer network 2531 provides access to the date file 2521 that may be shared between the computing devices 2541, 2551. Additionally, the computer network 2531 may be public or private and may be wired or wireless. The computing devices 2541, 2551 that are connected to the computer network 2531 may be any electronic device that is capable of connecting to a computer network and transmitting data over the computer network. Further, the computing devices are capable of receiving data for entry into a data file 2521 that may be associated with a country assessment.

Aspects of the invention may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. The plurality of modules may include a risk assessment module and a control assessment module. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments. For example, modules may be logically divided among various files and/or processors. Furthermore, one or more of the modules may be optional and may be omitted in accordance with various embodiments of the invention.

As illustrated in FIG. 26, a method of assessing an application may include identifying risk data that relates to the application, as in step 2601. As shown in step 2603, a risk score may be generated based at least in part on the risk data. The risk score may analyze, compile, report, and the like the risk data in any desirable manner.

The method of assessing an application may also include identifying control data, as illustrated in step 2605. As shown in step 2607, a control score may be generated based at least in part on the control data. The control score may analyze, compile, report, and the like the control data in any desirable manner.

The application may be assessed based at least in part on the risk score and the control score, as shown in step 2609. The assessment may be customized and may be changed as the needs of the line of business change. The assessment may be an algorithm that performs analysis and/or statistical calculations on the risk data and the control data. The assessment may also include a comparison of the risk score and the control score.

A threshold risk score may be identified. The threshold risk score may provide an adequate assessment of the risk data. The line of business may provide a range of threshold risk scores that may be adequate for an application's assessment. The threshold risk score may be quantitative and may include both objective and subjective criteria.

A threshold control score may be identified. The threshold control score may be predetermined by the line of business to provide an adequate assessment of the control data. The line of business may provide a range of threshold control scores that may be adequate for an application's assessment. The threshold control score may be quantitative and may include both objective and subjective criteria.

As described in detail above, the assessment of an application may be performed by a computer system. The computer system may operate over a computer network. The computer network may be any computer network that may be coupled to another computer network and may be capable of exchanging data between computing devices. The computer network may be the Internet or an intranet. The application may be accessible through a webpage on the computer, such as a webpage on the Internet or an intranet.

For example, a line of business may develop a webpage on their intranet for employee may access. The employee may be required to connect to the line of business's computer network through a direct connection within the network, a remote connection through a virtual private network (VPN), and the like. The employee may access the application through the webpage and store data in the data file through the application over the computer network.

The method of assessing an application may also include assigning a priority level to a user. The user may be anything that accesses the application including a human user, another application, a computing device, and the like. The priority level may be assigned to the user and may indicate a level of security to which the user may be privileged. The priority level may correlate to a human user's level of employment within the line of business. For example, a new or inexperienced employee may be assigned a low priority level that will prevent the new employee from accessing confidential or secret information relating to the application. In contrast, a senior employee may be assigned a high priority level and may be provided access to all of the confidential and secret information relating to the application.

The method of assessing the application may also establish a security. The security may include a password and may include encrypting the risk data and the control data. The security may also include encrypting a communication between a user and the application. Any desirable security may be used.

Another aspect of the invention is an apparatus for assessing an application that comprises memory and a processor. The memory may store a plurality of modules that comprise computer-executable instructions. The plurality of modules may include a risk data module that may be configured to store risk data relating to the application and a risk score module that may be configured to generate a risk score that may be based at least in part on the risk data. The plurality of modules may also include a control data module that may be configured to store control data that relates to the application and a control score module that may be configured to generate a control score that may be based at least in part on the control data. Further, the plurality of modules may include a comparison module that may be configured to compare the risk score and the control score.

The processor may be configured to execute the computer-executable instructions in the plurality of modules to generate a comparison based at least in part on the risk score and the control score. The comparison may include subjective and objective information and may be calculated by a predetermined algorithm or statistical analysis. The comparison may include any desirable information.

Yet another aspect, the invention may be a computer-readable medium comprising computer-executable instructions to perform a method. The method may comprise identifying a risk data that relates to the application and generating a risk score based at least in part on the risk data. Control data relating to the application may also be identified and a control score may be generated that may be based at least in part on the control data. The application may be assessed based at least in part on the risk score and the control score.

Aspects of the invention have been described in terms of illustrative embodiments thereof. A person having ordinary skill in the art will appreciate that numerous additional embodiments, modifications, and variations may exist that remain within the scope and spirit of the appended claims. For example, one of ordinary skill in the art will appreciate that the steps illustrated in the figures may be performed in other than the recited order and that one or more steps illustrated may be optional. The methods and systems of the above-referenced embodiments may also include other additional elements, steps, computer-executable instructions, or computer-readable data structures. In this regard, other embodiments are disclosed herein that can be partially or wholly implemented on a computer-readable medium, for example, by storing computer-executable instructions or modules or by utilizing computer-readable data structures.

Claims

1. A method of assessing a plurality of applications, comprising:

receiving at a risk analysis server having a processor first information identifying risk data relating to each of the applications, wherein the first information comprises, for each of the plurality of applications, a first indicator identifying whether the application is internal or external facing, a second indicator identifying a first subset of the plurality of applications upon which the application depends, a third indicator identifying a second subset of the plurality of applications that depend on the application, and a fourth indicator identifying a frequency of production code changes associated with the application;

generating by the processor a risk score for each of the applications based at least in part on the corresponding risk data for each of the applications;

receiving at the processor second information identifying control data, which includes objective binary information from a user comprising answers to predetermined questions regarding security controls for each of the applications;

generating by the processor a control score for each of the applications based at least in part on the corresponding control data for each of the applications; and

aggregating by the processor the risk data, risk score, control data, and control score for each of the applications into a database stored in memory communicatively coupled to the server comparing the risk score with a risk threshold and comparing the control score with a control threshold by determining and displaying the difference between the risk score and the control score;

plotting by the processor the plurality of applications into sectors according to the risk score and the control score of each of the plurality of the applications;

receiving at the processor third information answering at least one question relating to at least one application based at least in part on the information aggregated in the database for the at least one application;

assessing by the processor each of the applications based at least in part on at least one of the risk score, the control score, and the answer to the at least one question, and

assigning by the processor a user a priority level according to each user's employment position, wherein each user has access to the database based on the priority level;

wherein the control data comprise scalability information indicating whether or not each of the applications has adequate scalability to accommodate predefined future business needs.

2. The method of claim 1, further comprising generating a report for at least one of the plurality of applications that includes at least one of the risk score, the control score, and the assessment of at least one of the applications.

3. The method of claim 1, where the risk score is commensurate with the control score for at least one of the plurality of applications.

4. The method of claim 1, where assessing each of the applications includes comparing the risk score to the control score.

5. The method of claim 1, where the risk threshold is a range.

6. The method of claim 1, where the control threshold is a range.

7. The method of claim 1, where the plurality of applications is stored on a computer network.

8. The method of claim 7, where the plurality of applications is accessible through a webpage on the computer network.

9. The method of claim 1, further comprising establishing a security for accessing the database.

10. The method of claim 9, where the security includes a password.

11. The method of claim 9, where the security includes encrypting a communication between a user and the database.

12. An apparatus for assessing a plurality of applications, comprising:

a memory, storing by a processor at least one module comprising computer-executable instructions, the plurality of modules including:

a risk data module configured to store risk data relating to each of the plurality of applications, wherein the risk data comprises a first indicator of a peak transaction rate associated with each of the plurality of applications, a second indicator of a user access pattern associated with each of the plurality of applications, a third indicator of a percentage of application data associated with each of the plurality of applications periodically reviewed and certified as accurate, a fourth indicator of hours per week an average user works on each of the plurality of applications, and a fifth indicator of a volume of the application data stored for each of the plurality of applications;

a risk score module configured to generate a risk score for each of the plurality of applications based at least in part on the corresponding risk data;

a control data module configured to store control data for each of the plurality of applications wherein the control data comprises scalability information indicating whether or not each of the applications has adequate scalability to accommodate predefined future business needs and objective binary information from a user comprising answers to predetermined questions regarding security controls;

a control score module configured to generate a control score for each of the plurality of applications based at least in part on the corresponding control data; and

a comparison module configured to compare the risk score with a risk threshold and to compare the control score with a control threshold for each of the plurality of applications by determining and displaying the difference between the risk score and the control score;

a plotting module configured to plot the plurality of applications into sectors according to the risk score and the control score of each of the plurality of the applications; and

a processor operationally connected to the memory, the processor configured to execute the computer-executable instructions in the plurality of modules to generate a comparison for each of the plurality of applications based at least in part on the risk score and the control score to aggregate the risk data, the risk score, the control data, the control score, and the comparison for each of the plurality of applications into a database and to assign a user a priority level according to each user's employment position, wherein each user has access to the database based on the priority level.

13. The apparatus of claim 12, where the comparison module is configured to identify at least one of the plurality of applications that has a commensurate relationship between the risk score and the control score.

14. A non-transitory computer-readable medium comprising computer-executable instructions which when executed perform a method of assessing a plurality of applications, comprising: identifying risk data relating to each of the applications, wherein the risk data comprises a first indicator of a data growth rate associated with each of the applications, a second indicator of a first geographic location of data stores associated with each of the applications, a third indicator of a second geographic location associated with users of each of the applications, a fourth indicator of application patch management associated with each of the applications, a fifth indicator of a batch processing period associated with each of the applications, and a sixth indicator of regulatory scrutiny of business unit activities supported by each of the applications;

generating a risk score for each of the applications based at least in part on the corresponding risk data for each of the applications;

identifying control data for each of the applications wherein the control data comprise scalability information indicating whether or not each of the applications has adequate scalability to accommodate predefined future business needs and objective binary information from a user comprising answers to predetermined questions regarding security controls;

generating a control score for each of the applications based at least in part on the corresponding control data for each of the applications;

aggregating the risk data, risk score, control data, and control score for each of the applications into a database comparing the risk score with a risk threshold and comparing the control score with a control threshold by determining and displaying the difference between the risk score and the control score;

plotting by the server the plurality of applications into sectors according to the risk score and the control score of each of the plurality of the applications;

answering at least one question relating to at least one application based at least in part on the information aggregated in the database for the at least one application;

assessing the application based at least in part on at least one of the risk score, the control score, and the answer to the at least one question; and

plotting by the server the plurality of applications into sectors according to the risk score and the control score of each of the plurality of the applications.

15. The computer-readable medium of claim 14, where the risk score is compared to the control score for each of the plurality of applications.

16. The method of claim 1, further comprising identifying at least one of the plurality of applications that is associated with a line of business.

17. The method of claim 16, further comprising sorting the plurality of applications within the database based on the line of business associated with each application.