Just a year ago, we discussed the concept of Model Risk Management with you and how to tackle it using three lines of defense. The overall idea of Model Risk Management continues to grow and expand within a financial institution and we now bring you the key components for effective model risk management.
Today, many financial institutions consider model risk a subset of operational risk, as model risk mostly affects the team that creates and uses the model. There are various instances where financial institutions have gone through potential losses because of inadequate use of models and due to its criticality, regulators and financial institutions are treating model risk as a separate risk category within operational risk. The guidelines to do effective Model Risk Management were released by OCC in April 2011 as part of SR 11-7.
To adequately reduce risks inherent in bank models, a comprehensive and sustainable model risk management program should take into account a number of critical components, including the following:
1. Model Inventory:
You cannot effectively manage model risk without the overall knowledge and understanding of the models being used. It is imperative to begin this process with a model inventory. The inventory of existing models at any single financial institution may be large and can include both manual and automated processes and technology. Once a model inventory is completed, the organization should assess the enterprise risk of each model in the inventory in order to properly manage operational risks commensurate with the perceived risks in the model.
An important key aspect of model inventory is the accountability of models - meaning who is ultimately accountable for controls, compliance, and oversight of the models identified during the inventory. It is necessary to have adequate governance during the model reporting phase to make sure correct information is added with model definition. Few key examples of such information would be upstream, downstream model details, output/input variables, model type etc.
2. Model Development, Implementation & Use:
Once the model inventory is complete, the financial institution must put in the proper time and planning around model development. When financial institutions design models, relevant stakeholders – including executives, IT personnel, and affected line-of-business leaders – should assess the models for efficacy, evaluate the design structure, the approach to implementation, and the use of the model’s output. A plan should be clearly defined and followed on what all steps will be involved when developing a model; necessary documents should be attached by team owners during various model development cycles.
Once a model is developed, the model owner should also make sure the model is accurately implemented as expected.. Documentation of model implementation and its usage should also be done in a centralized model risk management system. According to the guidance, “An understanding of model uncertainty and inaccuracy and a demonstration that the bank is accounting for them appropriately are important outcomes of effective model development, implementation, and use.” Executives must analyze model results relative to limitations and design assumptions in order to assess the potential performance of a model before other stakeholders use the results.
3. Model Validation & Ongoing Monitoring:
Ongoing evaluation of models is required to confirm that results are accurate and controls are adequate. The guidance defines model validation as “the set of processes and activities intended to verify that models are performing as expected, in line with their design objectives and business uses. It also identifies potential limitations and assumptions, and assesses their possible impact.” While the concept of validation is not new, the guidance expands the expectations for an effective validation review. The guidance states, “All model components, including input, processing, and reporting, should be subject to validation; this applies equally to models developed in-house and to those purchased from or developed by vendors or consultants.”
Model assessment and validation is central to model risk management and its fundamental principal is effective, independent and effective assessment. Model owners should make sure models are appropriately scheduled for assessment and validation cycles based on model criticality. The frequency and intensity of each model should be considered based model risk, associated tier, model linkages, etc. The guidance defines “Effective model validation helps reduce model risk by identifying model errors, corrective actions, and appropriate use. It also provides an assessment of the reliability of a given model, based on its underlying assumptions, theory, and methods.”
The second core element of the validation process is ongoing monitoring. Such monitoring confirms that the model is appropriately implemented and is being used and is performing as intended.
4. Model Tuning and Optimization:
Tuning and optimization is the ongoing process of testing and enhancing the model. Model tuning is an important aspect of development and implementation of a model, as well as its ongoing management and sustainability. The team accountable to enforce model changes should make sure adequate reports and results are attached in the system that confirms recommended changes are implemented in the model. Again, there should be defined processes based on riskiness of model and intensity of changes during complete change log lifecycle.
Champion/Challenger is a well established methodology for validating the effectiveness of changes to Financial Institutions’ decisioning logic and risk policy. Champion/challenger testing always starts with a new idea- the challenger. It could be something small like a modified attribute. It could be something big like a whole new decisioning process for near-prime consumers. Once the challenger has been created, it needs to be tested against the champion (the current policy).
The final component of model risk management guideline is “Reporting”. Adequate reports enable various model stakeholders to achieve successful Model Risk Management. Reports with various assessment and validation details can help financial institutions provide more visibility on ongoing monitoring aspect. Enterprise wise health check reports can provide absolute portrait of models used in the organization, total risk carried due to models, details of critical models etc which can help better business decision and improved governance.
A robust model risk management program represents an opportunity for realizing tangible benefits that go beyond compliance with regulations. An effective Model Risk Management can significantly improve business decisions, a continuous increase in operational efficiency and a reduction in the financial and non-financial costs of remediating errors that go unnoticed.
The previous steps were designed to ensure proper model risk management for any financial institution. Is your organization doing it differently? Is it working or are you experiencing challenges in the process? I would love to hear from you and share ideas!
Garima Chaudhary is a Senior Sales Consultant for Oracle Financial Services Analytical Applications. She can be reached at garima.chaudharyAToracle.com.
The views expressed herein are the views of the author and not necessarily the views of the employer.