Annual validation confirms high performance levels for VantageScore 3.0 model

Date: June 25, 2020

The yearly VantageScore model validation process shows once again that the VantageScore 3.0 credit scoring model remains a highly effective consumer credit risk management tool in all three performance dimensions relevant to lenders today: predictive accuracy, universe expansion, and consistency.

VantageScore Solutions validates all its credit scoring models annually and is the only leading credit scoring model developer to publish its validation results. This year’s VantageScore 3.0 validation is available as a downloadable white paper.

The results confirm that the VantageScore 3.0 model remains highly predictive in the wake of the great recession, just as it did before and during the market downturn.

Publishing validation results also promotes accountability and greater transparency in the credit scoring industry and helps financial institutions with their model governance. Annual model validations took on even greater relevance after the Office of the Comptroller of the Currency (OCC) issued guidance in 2011, advising industry participants that rely on modeling and analytics for decision-making to develop more robust validation functions.

The validation process compares actual consumer default rates to those predicted by the VantageScore 3.0 credit scoring model. The process also benchmarks the model’s performance against commercially available, proprietary generic risk models provided by the three national credit reporting companies (CRCs), Equifax, Experian and TransUnion.

VantageScore 3.0 was validated using 4.5 million anonymized U.S. consumer credit files, randomly selected from the databases of each of the CRCs. The performance time frame was 2012 to 2014.

The validation shows that the VantageScore 3.0 model achieves material performance lift in all three key dimensions of the analysis:

Score predictive performance: Measures of score accuracy are highlighted by double-digit percentage increases in Gini coefficient, an industry-standard measure of score-performance improvement, in comparison with benchmark models. These improvements were seen in the upper portion (600 – 850) of the Vantage Score 3.0 scale range of 300 – 850. The upper portion of the range denotes consumers with the lowest credit risk and who are the most attractive to many lenders, in mortgage origination and account management scenarios. (Origination performance looks at default rates on loans issued three months or less prior to the validation performance time frame; account management looks at defaults on accounts more than 24 months old.) Results also demonstrate overall lift across three major credit industry segments: auto lending, bankcards, and mortgage.

Score consistency: More than 90 percent of the consumers whose files were used in the annual validation had credit files at all three of the national credit reporting companies. Within that group, 90.3% of the consumers had VantageScore 3.0 credit scores at all three CRCs that varied by no more than 40 points. (Every 40 points on the VantageScore scale represents a doubling (or halving) of the odds of default.) Moreover, the validation demonstrates that using the identical model across all three CRCs can reduce risk exposure and potentially increase the lending universe by as much as 11 million consumers for lenders requiring a credit score above 680 from multiple CRCs.

Universe expansion: Based on extrapolated results, the VantageScore 3.0 model continues to provide scores for over 35 million more consumers than conventional models can. The Gini coefficients associated with the VantageScore credit scores of these “newly scoreable” consumers also demonstrates a high level of accuracy.

Conventionally unscoreable consumers who may be scored with VantageScore 3.0 include:

  • New Entrants – consumers with trades that are six months or less old;

  • Infrequent Users – consumers who use credit but have not used credit within the most recent six month period of time, but who have used credit within the last 24 months;

  • Rare Credit Users – consumers who have not used credit in the last 24 months; and

  • No Trades – consumers with only public record and collections records.

“Credit scores are often the gateway to credit, and lenders also can use them as tools to expand their applicant pools without lowering credit standards,” said Sarah Davies, VantageScore senior vice president for analytics, product management and research. “Our validation shows that millions of consumers who are likely outside a lender’s purview can still be accurately scored. Their scores are reliable and can be used as part of lenders’ broader underwriting and marketing strategies, in accordance with safe and sound risk management processes.”