Webinar focuses on scoring-model governance

Date: June 25, 2020

Financial-services consultant Chris Carroll joins VantageScore Solutions SVP Sarah Davies for the latest VantageScore webinar, Converting to a New Credit Scoring Model: Regulatory Focus. The session details best practices for implementing a new credit scoring model, with a mind toward addressing regulatory concerns.

In the wake of economic downturns, Carroll explains, financial-industry regulators typically update guidelines for the institutions they oversee, with the goal of limiting vulnerability to similar conditions in the future. In the aftermath of the 2007–08 financial crisis, he says, automated scoring models came under regulator scrutiny for two reasons:

  • Lender reliance on automated models had been increasing in the years preceding the crisis.

  • In many instances, agencies’ guidance on scoring models and validations had not been updated for several years, and they were due for revision.

The webinar explains how to implement a new scoring model in a way that satisfies guidelines issued following the crisis, with particular focus on Supervisory Guidance on Model Risk Management, a document issued in 2011 by the Office of the Comptroller of the Currency (OCC) and the Federal Reserve Board of Governors.

Carroll offers tips on meeting the OCC guidance and offers a number of best-practices recommendations, including:

  • Board members and relevant senior managers should be able to explain the function of scoring models in non-mathematical terms.

  • Teams responsible for use of models must understand their inputs—the data they operate on to make their calculations. If any inputs are outputs from other models, teams must understand the source models’ behavior as well. (e.g., if an institution uses a VantageScore credit score as an input into a custom-built decision engine, the team should have a good understanding of the VantageScore model as well as the custom model).

  • There are exceptions to everything, so regulators will expect to see policies and procedures for addressing cases that fall outside the scope of a model.

  • When swapping in a new credit score model, back-end testing and calibration are important to ensure that the new model performs across your portfolio in a way that’s consistent with stated policies and strategies. Thorough documentation of these steps is also essential.

Davies next provides an overview of specific applications of the OCC guidelines to implementation of a new credit score model, touching on issues including:

  • Transparency: Understanding the conceptual soundness of a given model.

  • Materiality: How introducing a new scoring model affects the distribution of risk across a portfolio and whether that shift should impact the lender’s strategy.

  • Effective Challenge: Identify the proper testing protocols to ensure that scores provide minimal risk and maximum business opportunity, and define procedures for monitoring, analyzing and reporting to ensure ongoing transparency.

Next Davies presents a recommended sequence to follow when approaching the implementation of a credit score:

  • Use validations to understand whether or not the new score improves on the old score’s performance, in terms of predictive performance, the size of the scoreable universe or both.

  • Determine how the new model affects risk distribution within your population and, if appropriate, adjust your strategies accordingly.

  • Ensure that the model meets compliance and regulatory criteria.

  • Develop and implement protocols for testing, analysis and reporting of model performance to ensure maximum transparency.

Carroll sums up the session by noting the importance of creating a culture of communication, in which technical team members such as model developers find common language with business managers and senior executives. If those teams are able to converse comfortably about their goals and processes, he says, they should be able to confidently address any questions regulators may raise.

Following the webinar’s guidelines when implementing a new credit scoring strategy will ensure each institution demonstrates the four goals Carroll attributes to all financial-industry regulations: process, structure, discipline and accountability.