Credit Score Stability in a Changing Risk Environment
July 2009
Overview
As the economy shifts, the consumer risk levels indicated by credit scores shift as well. This is as natural as the variance in mileage seen in a car when operated under varying driving situations. Although the EPA can rate the mileage performance of a particular automobile, the mileage will vary on different road conditions. Credit scores shift their meaning with changing environmental conditions as well.
In the context of credit score performance, 'environmental conditions' are defined as substantive changes in the ways in which consumers manage their debts. Exotic mortgage products, high risk payment strategies coupled with unsound underwriting methods such as unverified income, inflated appraisals, etc., have primarily created and driven these changes to the environment. Such changes have the potential to alter any credit score's ability to predictively identify those consumers who will pay their debts on time and those who will not. With the recession that began in December 2007, an extreme deterioration in consumer debt management has been observed in loan originations, manifesting in record-high real estate foreclosure levels, especially within the hardest hit states such as California. As consumer debt management behavior patterns are reflected in the environment, many questions are raised regarding the effectiveness of credit scores and their ability to separate higher credit quality from poorer credit quality consumers. Lenders relying on credit scores in any part of their business need to regularly assess whether or not the credit score they are using is adequately capturing these changing conditions and further determine if any resulting recalibration of the score is needed in order to align their risk management strategies with new consumer debt management behavior.
This paper applies best-practice analyses on credit scores to assess levels of continued performance for both account originations and existing account management in light of significantly deteriorating conditions. A critical review is undertaken on real estate loans with emphasis on those states most impacted by these foreclosure trends. Additionally, performance reviews are conducted for credit card and auto loan portfolios to determine whether similar deteriorating behaviors are appearing in these industries.
Throughout the paper, VantageScore¨ is used as the benchmark score. Important questions that are answered include:
- Does VantageScore appropriately capture the changing consumer risk profiles?
- Does VantageScore continue to separate higher credit quality ('good') consumers vs. poorer credit quality ('bad') consumers?
- Does VantageScore continue to effectively separate good vs. bad consumers in the real estate industry?
- Does VantageScore effectively separate in the states that have been severely impacted by the credit deterioration?
Consumer behavioral trends and observations are also reported.
Summary Conclusion
Through annually updated performance charts, VantageScore reflects the increased consumer credit risk currently experienced across all industries in this severe national economic downturn. VantageScore's predictive performance in loan origination and existing account management for all industries remains strong, but this is demonstrated specifically for the real estate industry. These same results are provided for California where foreclosure rates are at record levels.
The score remains highly predictive in its ability to identify low- and high-risk consumers, demonstrated by its' rank ordering strength.
These findings are especially significant given default rates in California have increased by as much as 900 percent.
Background
Credit score models are built using patterns of consumer debt management characteristics and payment performance over time within the context of particular economic environment.
A pattern consists of characteristics and performance. Characteristics are developed from tradelines reported by lenders and contained in consumers' credit files. Each characteristic is designed such that it provides predictive insight into how consumers pay their debts. Examples of characteristics include number of loans, type of loans, number of loans paid on time, loans paid late, severity of late payments, size and age of loans, etc. VantageScore performance is defined as whether the consumer paid their debts on-time or allowed one or more debts to become 90 days or more past due, (defined as default propensity). For the purposes of score design and use, consumers who allow a debt to become 90 days or more past due are referred to as "bad" while consumers who are never more than 30 days late on a payment are labeled "good."
To build a robust and representative model, millions of patterns are required to capture the diverse debt management behaviors of the population. For the twelve scorecards contained within VantageScore, patterns for 15 million randomly selected consumers representing the U.S. population from 2003 to 2005 were used to build the model. Each consumer pattern is evaluated in the context of all other patterns and then assigned a value, called the credit score. The consumer's credit score identifies the place, or rank, of that consumer within the overall pool of consumers according to the likelihood that the consumer will allow one of their debts to become 90 days or more past due. Consumers who are less likely to become 90 days past due on a debt receive a higher score or rank, those more likely to become 90 days or more delinquent receive a lower score or rank.
Credit scores are presented in bands, typically each band is a range of 20 points that are aligned with a particular default propensity rate, also often called 'good/bad odds' rate. Default propensity is defined as the likelihood that a consumer will allow a debt to become 90 days or more late. Good/bad odds are defined as the number of 'good' consumers that obtain a score in the risk band for every 'bad' consumer obtaining a score in the same risk band. These rates are based on the overall population that was used to calibrate the score. Note that the rates reflect the general debt repayment profile of the population in response to all debts, rather than the specific repayment behavior of consumers for a specific lender. Consequently, they should not be viewed as an absolute probability of default for a particular lender. However, the relationship of the default rates between score bands provides meaningful insight into the change in risk that consumers reflect as score bands change.
This information is provided to the lenders in the form of a table called a performance chart or odds chart. (See abbreviated chart below).
Odds charts can be graphically depicted to provide a general risk profile of a population ranked by a particular score. The overall position of the cumulative risk profile against the Y-axis reflects the overall risk associated with the sample. If a score provides effective rank ordering, then the interval risk profile should show monotonically increasing risk rates as the score reduces. The example at left contrasts a score that is effectively rank ordering – interval risk rates are increasing monotonically (blue line), with a score that is failing to rank order (red line). In this example, the red line of the failing score shows several score ranges that have interval risk rates that are higher than risk rates at lower score ranges. In other words, the score is assigning higher scores to more risky consumers. Consequently a lender could experience greater risk exposure using this score. Note that when this effect occurs at higher credit score ranges, it is less concerning given the associated risk is very small.
Ultimately, lenders must evaluate the impact of reduced separation strength in the context of an overall P&L that incorporates the specific risk characteristics of the lender’s business, origination and account management strategies. If the score fails to rank order accurately at key score cut-off zones resulting in losses with no offsetting incremental revenue for the lender, then it may be necessary to consider either score redevelopment or using an alternate score.
Changes in consumer risk are captured and reflected in the annual updates of these performance charts. Naturally, updated performance charts better reflect current consumer risk and are recommended so that lenders can benefit from more predictive risk insights enabling prudent strategy design. Risk profiles from performance charts for the last four validation periods for VantageScore are presented below.
Finally, evaluating cumulative risk profiles over time provides an understanding of how the overall consumer risk distribution changes over time.
During the 2003-2005 timeframe, given the economic and credit environment, consumers with a credit score of 750 had a default propensity of 0.4 percent. Performance charts calibrated on the most recent timeframe, June 2006 to June 2008 capture the credit quality deterioration driven by the real estate industry of the last four years. Consumers with a credit score of 750 now have a default propensity of 2.4 percent. It is likely that score cut-offs in risk management strategies may need to be adjusted upward in order to reduce the lenders risk exposure given current risk conditions.
The Study – Ranges on Score Cutoffs
Score predictiveness and stability is compared using performance charts and underlying consumer data used in the VantageScore credit score model. Four consecutive two-year time periods are reviewed: June 2003 – June 2005, June 2004 – June 2006, June 2005 – June 2007, and June 2006 – June 2008. These time periods align with the annual revalidation studies undertaken by VantageScore since its development in 2005, which used data from the June 2003 – June 2005 timeframe. The data for the study was provided by TransUnion.
Rank ordering performance charts presented in the study reflect the score ranges that focus on primary risk tiers and score cut-offs for subprime, near prime and prime. These ranges were determined by calibrating VantageScore with the OCC/OTS Mortgage Metrics report1. Delinquency and derogatory metrics for nine banks and five thrift institutions were sourced from the OCC/OTS Mortgage Metrics report. Metrics were captured at the overall level and at the prime and subprime levels. A credit file dataset was built on commensurate banks and timeframe with the same delinquency and derogatory profile. Each consumer was scored using VantageScore and rank-ordered. The cumulative risk level was calculated. Aligning the risk level in the VantageScore portfolio with the metrics from the OCC report shows that the VantageScore prime score cut-off is 700, near prime score range occurs between 641 - 699 and subprime score cut off is 640 and below.
Default rates for the four time periods referenced above are presented in the form of bar charts for the entire population (score range 501 to 990). Additionally, trends in mortgage loan size and payment patterns are offered in the section titled ‘Underlying consumer behavior trends.’ These data are offered in order to provide transparency into underlying consumer behavior and to demonstrate the alignment of this behavior with the score’s ability to separate good and bad performance.
Vantage Score Stability for Account Origination
VantageScore‘s performance is evaluated when used for underwriting and loan origination strategies.
Summary Insights
The recent changes in consumer debt management behavior are clearly captured in the risk profile for VantageScore when revalidated using consumer data from June 2006-June 2008. The profile shows an overall increase in consumer risk for the entire population captured in the upward shift of the risk curve, signaling the likelihood of fewer ‘good’ consumers for each ‘bad’ consumer. Additionally, the slope of the curve has become marginally flatter, particularly for higher credit quality consumers – prime and super prime. Despite the slight change in the slope, the rank ordering performance continues to remain very strong throughout all consumer risk tiers providing lenders with excellent insight into progressively increasing risk across score bands.
These changes in risk profile are driven predominantly by the real estate industry with only a minor contribution from increased risk in the auto industry. Debt management behavioral trends for the real estate industry over the last four years are presented in the ‘Underlying Consumer Behavior’ section. These trends show how exotic products (represented by larger loan size), unsound underwriting and repayment strategies combined to result in widespread and severe losses.
Despite the increase in consumer risk on real estate loans, VantageScore remains highly predictive for the industry, including the states that have experienced severe increases in foreclosure rates.
Lenders are advised to update their account origination and management strategies to reflect the incremental risk associated with their score cut-offs by using current performance charts.
Score Stability for Originations
Removing real estate defaults from the risk profiles shows that the Jun06Jun08 profile is now very similar to prior years.
Consequently, we can conclude that real estate industry is the primary contributor to increased credit risk revealed in the profile.
Key Industry Review – Real Estate Origination
The effects of increasingly high-risk mortgage products offered to unqualified consumers resulting in growing default levels are captured in the risk profile depicted in the performance graph below. While the entire risk profile for underwriting in this industry has substantively deteriorated (observed by the rise in defaults on all score bands), VantageScore continues to rank order consumers correctly, allowing lenders to continue to identify higher credit quality consumers from among poorer credit quality consumers.
Real Estate Originations: Rank Ordering Performance
While risk levels are clearly higher overall, VantageScore rank orders effectively as seen by the monotonically increasing bad rate interval values in the chart below. The study below focuses on primary cut-off score ranges, 590-930. (Note: VantageScore’s full range is 501 to 990.) Consumers are scored and rank ordered by deciles.
The bad rates monotonically increase by score range, providing the appropriate insight into population risk to facilitate judicious underwriting and loan originating strategies.
Real Estate Account Origination – California
The analytics presented for real estate at the overall portfolio level are repeated for California where record foreclosure rates have been experienced. The objective of these analyses is to determine whether VantageScore can be effectively used for risk assessment on mortgage originations in areas where default levels have increased far beyond previous levels.
As observed, the score reflects the changes in the consumer risk environment and consistently delivers predictive rank ordering.
The performance chart above shows that VantageScore reflects the increase in consumer defaults (over 900 percent increase in 90+ days-past-due rates) in California real estate originations between the June 2003-June 2005 timeframe and the June 2006-June 2008 timeframe. VantageScore’s rank ordering performance during the recent extreme default levels remains strong, as shown below.
Bankcard Originations
Overall default levels have increased by approximately 25 percent, reflected in the profile shift, from June 2003-June 2005. This increased risk is reflected in consistent measure throughout the score range. As a result the June 2006 – June 2008 risk profile is similar to the June 2003 – June 2005 risk profile, indicating similar predictive strength. Given the increased default rates for bankcard in 2009, close attention will be paid to performance in this industry in the 2007-2009 validation.
Auto Originations
Default levels for auto trades have increased approximately 30 percent when compared to the June 2003-June 2005 timeframe, again captured by the upward shift in the risk profile for the June 2006-June 2008 period. The interval rates continue to rank order effectively.
Score Stability for Existing Account Management
Summary Insights
At the overall portfolio level for existing account management, VantageScore remains highly predictive despite the effects of the increased default levels. The incremental risk in consumer behavior, predominantly related to real estate, is reflected by the default rates assigned to the score through annual validation and performance chart publication.
Lenders should update their account management strategies to reflect the incremental risk associated with their score cut-offs by using current odds charts.
Despite this increase in risk, the score has maintained its rank ordering strength, providing confidence that lenders are able to predictively assess consumer risk levels for all industries.
Score Stability for the Overall Portfolio
VantageScore captures the incremental risk at each level of consumer credit quality, reflected in adjusted risk estimates by score range and monotonically increasing risk as the credit score becomes lower, above right. Removing the real estate industry and recreating the performance charts shows that the risk profile for June 2006-June 2008 reflects a near identical risk profile to prior years at the overall portfolio.
One can conclude that the observed shift and slope change are driven almost exclusively by increased failure to repay real estate loans.
Key Industry Review – Real Estate Existing Account Management
As observed in the odds chart at left, there is a major shift in the default profile between the June 2006-June 2008 timeframe and prior timeframes. The shift is generally consistent for all risk tiers.
Real Estate Rank Ordering Performance
Given the substantive change in risk profiles for the real estate industry, a review of VantageScore’s ability to properly rank order was undertaken. The result is effective rank ordering demonstrated by interval bad rates that monotonically increase as score ranges lower.
Real Estate Existing Account Management – California
The analytics presented for real estate at the overall portfolio are repeated California. The objective of these analyses is to determine whether VantageScore can be effectively used for risk and credit management in these states.
As observed below, VantageScore captures the changes in the risk environment and consistently delivers predictive rank ordering.
The graph captures the experienced credit deterioration in California, reflected in the state-wide default rates. Rank ordering remains strong, shown below.
Bankcard Existing Account Management
The risk profile for the bankcard industry has remained consistent over the last four validation timeframes. A slight slope change is exhibited in the most recent validation period in line with the increase in defaults, as seen below left. The score’s rank ordering ability remains strong.
Auto Existing Account Management
As with bankcard, the risk profile for the auto industry has remained consistent. Rank ordering remains strong.
Underlying Consumer Behavior Trends
The VantageScore performance analytics presented previously highlight three fundamental shifts in consumer payment behaviors:
- A substantive increase in consumer defaults occurred in the June 2006-June 2008 timeframe, most significantly experienced in loan origination
- The primary driver of the deterioration is real estate loans
- The increase in consumer defaults, initially observed in subprime, has expanded into prime and super-prime tiers.
A review of underlying consumer trends was conducted to ensure that VantageScore captures consumer behaviors appropriately.
Randomly selected real estate loans, originated over a 12-month window, were collected and profile statistics developed for trend comparisons. Portfolios were created for loans originated from June 2002-June 2003, June 2003-June 2004, June 2004-June 2005 and June 2005-June 2006, respectively. Profile statistics were developed for each portfolio over a two-year performance window, i.e. performance over the June 2003-June 2005 timeframe for the loans originated in June 2002-June2003 window was used to develop profiles and trends. Loans were categorized by VantageScore risk tiers, subprime (640 and below), near prime (641 to 699), prime (700 to 899), super prime (900 and above).
Average Loan Size
While subprime loan size has remained relatively constant across time, loan size for higher credit quality consumers has increased dramatically. In June 2002-June 2003 originations, prime loans were an average of 1.23 times the size of subprime loans and grew to 1.48 times the size of subprime loans by June 2005-June 2006. Similar trends in loan size are observed for super-prime.
Clearly, ‘exotic’ mortgages that combine: (a) - increasing loan size at higher credit quality levels, with (b) - reset triggers generating significantly higher monthly payments, are a primary contributor to the rapidly rising default levels over the last several years.
Early Delinquency Flow
Thirty-day delinquency rates show universal deterioration but most significantly in the near-prime and prime credit tiers, with the major downturn beginning in the June 2005-June 2007 window. For example, prime loans in June 2005-June 2007 (05/07) were 78 percent more delinquent than in the June 2003-2005 (03/05) window.
Derogatory Flow
Derogatory events are defined as charge-offs, foreclosures and bankruptcies. Again, the universal impact of high risk mortgage products and weak underwriting criteria is observed at all credit tiers. Although rates for prime and super-prime are much smaller in absolute terms, the deterioration in credit quality is occurring at a much greater rate.
State-level Comparison
State-level loan amount, delinquency and derogatory metrics were indexed to the equivalent metrics at a national level, providing the clarity as to how and where the deterioration in real estate loans manifested.
Conclusion
The payment trends observed in these analyses show the effects of the real estate industry deterioration on overall consumer debt management behavior.
VantageScore has captured and reflected these effects in score performance charts used by the lending community. Despite the changes in consumer risk, the VantageScore model retains strong rank ordering power, providing confidence in its application for new and existing account management.
VantageScore Solution, LLC continues to monitor these trends and provide marketplace updates regarding the performance strength and stability of the VantageScore algorithm.