The completion of the 2014 validation process for VantageScore 3.0, the first validation since the model was introduced in March 2013, affirms the model’s value as an extremely effective risk management tool for lenders. Validations for the VantageScore 1.0 and 2.0 models also confirmed their ongoing effectiveness as risk-management tools.
Validation is a process for testing the effectiveness of a credit scoring model’s primary function: predicting the likelihood that a borrower will default (go 90 days or more overdue on a loan payment within 24 months). Validation entails looking at actual consumer data (minus any personal information such as names, addresses, social security numbers, and account numbers) for a 24-month period and comparing the model’s default predictions to actual consumer default behavior recorded for that period.
The VantageScore 3.0 validation used the time period from June 2011 to June 2013, providing an accurate representation of recent consumer behaviors in the current credit environment. The validation methodology followed recommendations issued in 2011 by the U.S. Office of the Comptroller of the Currency in an update to its “Supervisory Guidance on Model Risk Management.”
The validation looked at three distinct populations in terms of credit risk:
- The top 70 percent, a population segment representative of many mainstream lender portfolios, with scores in the 600-850 range (using the VantageScore 3.0 scale of 300-850);
- Lenders’ primary “risk decision zone” — the middle 40 percent of the population in terms of risk, a segment in which many lenders place their minimum score cut-offs — comprising consumers with VantageScore 3.0 scores from 600-770; and
- Subprime consumers, with VantageScore 3.0 scores below 600.
Results were benchmarked against those of generic credit scoring models from each of the three national credit reporting companies (CRCs) — Equifax, Experian and TransUnion — and against earlier versions — VantageScore 1.0 and 2.0 — of the VantageScore model.
Among the highlights of the results:
- The VantageScore 3.0 model outperforms benchmark models by an average of 17 percent among consumers in the top 70 percent of the credit scoring range.
- Among consumers in the risk decision zone, VantageScore 3.0 outperforms benchmark models by an average of 33 percent.
- VantageScore 3.0 continues to strongly rank order consumers across the economic spectrum and geographic ranges, regardless of the extent to which regions are impacted by unemployment and home price depreciation.
- The VantageScore 3.0 model continues to deliver highly consistent credit scores across all three credit reporting companies, with 90 percent of consumers receiving a credit score within 40 points, a spread that represents a doubling or halving of probability of default.
- VantageScore 1.0 and VantageScore 2.0 also continue to demonstrate a high level of accuracy and consistency, and both models also remain exceptional risk management tools.
“We are extremely pleased with the results of our model validations. Today, more than ever, lenders need assurances that the credit score model they use delivers the highest level of performance possible,” said VantageScore Solutions CEO Barrett Burns. “This information is particularly important from a model governance and transparency standpoint, as regulators are keen to see for themselves how a lender’s risk models perform.”
Detailed results of the validation are available via white papers posted at VantageScore.com.