Learning about Machine Learning

Date: July 28, 2021

Attend any conference or discussion about the future of consumer credit decisioning and the topic of Artificial Intelligence (AI) and Machine Learning (ML) is sure to be mentioned.

We’re often asked to opine on this topic as well because we used ML in the development of our latest model…but we’ll get to that in a second.

What I observe is that we all have different definitions of these terms and sometimes the context doesn’t really make sense. For example, what I often see is that a data scientist’s end goal is to use AI or ML; when, in actuality, they are tools…AI and ML are a means to an end and often in today’s environment that end is to make better and better credit decisions for a larger and larger population. These goals must be balanced with a thorough review and testing to ensure the outcomes are in no way causing unintentional harm towards any group.

We recently responded to an interagency Request for Information (RFI), jointly published by the Treasury (OCC), Fed, FDIC, NCUA and CFPB, for information and comment on this topic. Specifically, the request was for “Information and Comment on Financial Institutions’ Use of Artificial Intelligence, Including Machine Learning”. I thought I’d use our newsletter to answer a few fundamental questions about this topic and excerpt from the letter.

What is machine learning?

    …we view Artificial Intelligence (AI) as a broad field, generally defined as machines mimicking some form of human behavior. Machine Learning (ML) is a sub-discipline within the field of AI which is characterized by the use of algorithms that are able to mimic how humans learn, thereby improving with experience and new data. Many of the standard tools used for classification, regression, clustering and detection analyses are ML algorithms and have been around for a long time.

    More recently, a more complex subset of ML algorithms has been put into broader use, fueled by the increased availability of computing power. These more complex methods, sometimes referred to as “Deep Learning,” include neural networks, natural language processing and image processing. The recent discussions related to AI and Machine Learning tend to center around these more complex methodologies, focusing on issues related to transparency, explain-ability and the potential “black box” nature of some of these methodologies.

    What are the benefits of using these techniques in the consumer lending industry?

    Modern AI and ML techniques have the potential to provide significant benefits in consumer lending by providing increased accuracy in projections and estimates, deeper insights to solve complex problems, as well as significant efficiency gains and transferability of applications. At the same time, consumer lending is a highly-regulated industry with strict requirements regarding fair lending and fair credit reporting related consumer protections, as well as the need for safety and soundness controls related to risk management systems and the models that are used in decisioning. Therefore, any potential application of AI and ML tools must appropriately take into account these expectations and requirements, and weigh the potential the benefits of these tools against potential risks and the principles of transparency, explain-ability, consistency and fairness. Further, the use of advanced methods cannot be a substitute for domain expertise and sound business judgment. The resulting algorithms, underlying assumptions and results must be carefully reviewed and challenged by subject matter experts to ensure that business, policy, legal and control objectives are met.

    How did VantageScore use machine learning?

    In VantageScore 4.0, our data scientists have incorporated ML-based approaches as part of the data exploration and model development processes to augment the development of credit data attributes in consumer segments for which there is limited credit history information available…Our data scientists identified the highest performing multi-dimensional relationships that came to light using ML, as well as their common elements. They then translated these elements into traditional, structured attributes which could be incorporated into the model development process. This translation was done to ensure that consumers could understand the behavioral causes of the credit score they received and what behavioral changes would assist in improving their credit score…The benefits of a balanced approach which leverages ML combined with strong domain expertise has become very apparent in the case of VantageScore. We observe that VantageScore 4.0 provides a significant lift in predictive performance, particularly for consumers who have been recently inactive with credit but who have accounts on their credit files. There are approximately 24 million such consumers (out of the 37 million consumers scored by our models but are otherwise unscoreable by a legacy scoring models) who have not actively used credit within the last six months. The ML algorithms allow deeper insights about these consumers for incorporation into the model. In return, these deeper insights help these consumers to receive a fair and accurate credit score that is more reflective of their creditworthiness and facilitates their access to mainstream credit products best suited to their needs.

    To be more specific, VantageScore scientists developed tens of thousands of multi-dimensional combinations of data elements that were analyzed to examine sparse credit histories. A limited number of these predictive data relationships met our specific criteria and were included as attributes in the model, yielding a 10 percent performance lift among those with “dormant” credit histories (i.e., consumers who have scoreable trades but have had no update to their credit file in the last six months) and more than a 30 percent performance lift amongst those with “No Trades,” as compared with the performance generated by legacy credit scoring models.

    We developed VantageScore 4.0, leveraging machine learning techniques and trended credit data, back in 2016 and launched it in 2017. Years later, we’re seeing a lot of lenders now using the model which is satisfying to be sure. What’s more: we also are seeing a lot of other developers exploring how to use AI and ML in a safe and sound way, which is also gratifying.

    To summarize: competition makes us all better! (and on a similar note: be sure to read The Financial Times article in this newsletter on how VantageScore is challenging the status quo)

    Please read on for some great articles...

    Best,

    Barrett Burns

    CEO and President, VantageScore Solutions