In 2011, the Office of the Comptroller of the Currency (OCC) expanded previously issued guidance that, in part, emphasized to lenders the importance of validating the models they use.
As part of its guidance, the OCC explicitly recommends that financial services firms utilizing predictive models and decision analytics perform regular validations to gauge model efficacy. The OCC’s guidelines apply both to lenders that utilize proprietary models and, importantly, to those that use vendor-generated models where there might not be the same transparency and understanding of how the model was built.
Two years after the guidance was issued, VantageScore Solutions commissioned a survey of lenders by SourceMedia Research, LLC to better understand the extent to which model validation is an annual practice.
The first issue the survey explored is the types of models lenders are using. Nearly 30 percent of lenders said they use more than one credit score model, and nearly 30 percent also said they use more than one vendor-provided credit scoring model.
Today there are dozens of models available, from multiple vendors — reflecting lenders’ preference for choice in the credit scoring marketplace. In fact, 69 percent of lenders said they want access to credit score models from multiple, qualified developers.
Another growing trend is lenders’ creation of their own proprietary or custom models. More than 40 percent of lenders said they use proprietary credit scoring models.
Widespread usage of credit score models throughout the lending industry clearly underscores the importance of validation. The survey found that 40 percent of lenders validate or test their credit score models annually. Smaller lending institutions are performing annual validations to a lesser extent than larger ones as the chart below demonstrates:
Model validation is a critical activity to verify that credit scorecards continue to work as intended and that model usage is in line with business objectives and expectations. A regular model-tracking and validation process can ensure that consistent and optimal model-based decisions are being made. It can also serve as an early warning system for identifying when a change may be necessary, whether it be an adjustment to a score cut-off strategy or a full model redevelopment.
Visit the VantageScore website for additional information about validation and testing of credit score models.