top of page
roberto-junior-4fsCBcZt9H8-unsplash.jpg

Optimizing Call Payment Results 

Analyzing and Predicting Payment Behaviors

In the world of finance, reaching out to customers at the right moment can make a huge difference. This is exactly what I aimed to uncover in my project with Preferred Credit Inc. (PCI). Just as the restaurant industry had to adapt and understand its recovery post-pandemic, I was curious to explore how financial institutions could improve their engagement with customers, especially through phone calls.

Data

My project harnessed PCI's extensive data on customer calls, payments, and outcomes. This data isn't just a collection of numbers; it's a treasure trove of insights waiting to be discovered. From when a customer was called, to the response received, every detail could potentially lead to a better understanding of how to approach them in the future. The analysis spanned over a significant period, allowing us to track patterns, successes, and areas for improvement in customer communication.

 looked into the timing of successful calls, the impact of various factors like the day of the week, and even broader economic indicators such as unemployment rates. These factors aren't just background noise; they play a crucial role in shaping customer behavior and, consequently, the success of PCI's interactions with them.

Findings

Through meticulous comparison and analysis, several key insights emerged:

  • Timing is Everything: Just as with restaurant patronage, the success of a call can hinge on when it's made. My analysis pinpointed the best times for making calls that are more likely to result in positive outcomes.

  • Economic Indicators Matter: Similar to how the pandemic's impact on restaurants varied by region and was influenced by factors like vaccination rates, the financial behavior of PCI's customers also showed sensitivity to broader economic conditions. For instance, periods of high unemployment correlated with changes in payment behaviors and call success rates.

  • Adaptation is Crucial: PCI can use these insights to tailor their approach to customer interactions, optimizing call times based on data-driven insights and adjusting strategies in response to economic conditions.

This project isn't just about making more successful calls; it's about understanding and adapting to the complex web of factors that influence customer behavior. We can appreciate the value of data analysis in navigating challenges and seizing opportunities in the financial sector.

The project is structured into two integral parts. The first part is an exploratory data analysis (EDA), which involves a thorough examination of the payment and call datasets. This stage is pivotal in understanding the underlying patterns and trends in the data, which inform the subsequent modeling process. The second part is the development of the Call Outcome Model. This predictive model leverages the insights gained from the EDA to accurately forecast the outcomes of calls made to customers regarding their loan payments.

Part 1: Exploratory Data Analysis
1. Exploring Payment Behaviours

This line graph titled "Monthly Trend of Payment Amount" tracks the progress of payment collections from January 2021 through January 2023. A clear upward trend in the total payment amount is evident, indicating an increase in the volume of payments processed over time. There are observable fluctuations month to month, with occasional dips that may signify seasonal variances or cyclical payment behaviors. The overall positive trend suggests growing effectiveness in payment collection strategies or possibly an improving economic climate that enables better financial capacity among customers. Each point on the graph represents a monthly aggregation, providing a straightforward visual narrative of the company's financial inflows.

Curiosity arises regarding the underlying reasons for this upward trend in payments. One might question whether it is simply a result of issuing higher loan amounts, thereby inflating the monthly payment figures. To address this, the next plot will examine the monthly payments as a percentage of the total loan amount, providing insight into whether the increase in collected payments is proportionate to the loan amounts disbursed.

The "Monthly Trend of Payment Amount % of the Total Loan Amount" graph shows a different story than the previous total payment trend. It began with a sharp spike in early 2021, where a high percentage of the loans were being paid off. This peak quickly flattens out, indicating that after an initial surge, the payment amounts leveled off in proportion to the total loan amount. This pattern suggests that the increase in payment amounts we saw earlier may be due to factors other than just higher loan issuance, such as improved payment collection or consistent repayment behavior over time, or potentially the company’s strategic shift towards issuing loans to less risky customers, leading to a lower percentage of the total loan being paid each month.

Slide5.jpg

To further explore the patterns, we've decomposed the monthly payment amount percentage. The trend component shows a marked decrease, suggesting payment percentages are falling over time. Seasonality shows payments peaking in April and then dipping, reaching the lowest point around August and September. The residual plot doesn't show any significant outliers, indicating that the trend and seasonal factors are the main drivers of payment percentage changes.

Slide6.jpg

In these graphs, I've explored the relationship between principal and interest contributions to total payments each month. The top blue graph shows the percentage of total payments that went towards the principal, and the bottom orange graph shows the interest percentage. There’s a stark contrast between the two: as the principal contribution starts high and then steadies, the interest contribution mirrors this with a sharp drop and then levels out.

What’s intriguing is that both lines seem to stabilize after the initial changes, suggesting that the ratio between principal and interest doesn’t vary widely month to month. It seems that, for most payments, the distribution between principal and interest remains fairly consistent, or that any changes in the distribution are relatively small and don't significantly impact the overall trend.

Slide7.jpg

This line graph represents the Monthly WRITEOFF Loss Amount Trend, showing the changes in the sum of money deemed non-collectable from closed accounts over time, without including canceled loans. There’s a pronounced peak in the middle of the timeline, suggesting a period where losses were particularly high. Following this peak, there’s a significant drop, indicating a period of recovery where either fewer accounts were written off as noncollectable, or the amounts being written off were smaller. The downward slope suggests an improvement in either the company’s collection strategies or customer repayment behavior.

The original data depicted in the top graph reveals a mid-year swell in Writeoff amounts before a downward trajectory. The trend line in the second graph echoes this pattern, climbing before a noticeable decline, suggesting an overarching reduction in the losses over time.

The seasonal fluctuations are pronounced, with a peak in Writeoff losses around April. This might correspond with financial events such as tax season impacts or the culmination of a fiscal quarter. The decrease post-April is consistent, bottoming out in October. This pattern might be influenced by business cycles or seasonal credit activities that affect customers' payment behaviors and the subsequent need for account write-offs.

In the residual component of the graph, all points are above the zero line, indicating that the observed Writeoff losses were consistently higher than what the model (moving averages) predicted solely based on the trend and seasonal components. This suggests there are additional factors contributing to the Writeoff losses that aren’t captured by the trend or seasonal patterns.

The forecast suggests that with each successive month, there will be a further decrease in the WRITEOFF loss amounts. The negative numbers illustrate that the rate at which losses are reducing is expected to continue. Essentially, this trend points toward ongoing improvement in minimizing WRITEOFFs over the projected period

image.png

The line graph depicts Average Delinquency by Age, showing fluctuations in delinquency across different age groups. Delinquency remains relatively stable with some variability up to age 90, where we see a significant spike, indicating a much higher average delinquency for this age group. This could be due to a smaller sample size or specific challenges facing the oldest customers.

2. Call Collection efforts Analysis

image.png

This bar chart depicts the number of calls made to different age groups. The 25-34 and 35-44 age brackets received the highest number of calls, with a slight decrease in calls to the 45-54 and 55-64 age groups, and the fewest calls made to the 18-24 and 65+ groups. This distribution may suggest a focus on age demographics that are potentially more likely to yield successful collections or may reflect the age distribution of the customer base.

This graph shows the Success Count and Average Success Rate by Month for call collections. It's clear that the number of successful calls, represented by the blue line, is generally on an upward trend, indicating an increase in the total number of successful collections over time. However, the success rate, marked by the red line, shows a slight downward trend.

The increase in successful call count despite the small dip in success rate could imply that the call center is making more calls overall. This suggests that the collection efforts are scaling up, and although the efficiency per call might be experiencing a slight decline, the overall collection amount is likely increasing. This is often a sign of an aggressive collection strategy that, while possibly reducing efficiency, is successful in recovering more funds overall.

This chart illustrates the Success Count and Average Success Rate by Day of the Week. The blue line representing the success count dips slightly mid-week before rising towards the end of the week, suggesting that more successful calls are made on Mondays and Fridays. The success rate, depicted by the red line, remains relatively stable across the week, with minor fluctuations, indicating that the rate at which calls are successful is consistent regardless of the day. This consistency in the success rate could reflect a steady approach to call quality or customer responsiveness throughout the week.

In this chart tracking call success from 7 a.m. to 8 p.m., the success count varies, peaking around midday and dipping in the late afternoon. Interestingly, despite these fluctuations, the success rate remains comparatively steady throughout the day, with a slight uptick in the evening. This could imply that while call center activity—and perhaps customer availability—fluctuates, the effectiveness of calls in terms of success rate is consistent, even improving slightly by the end of the day.

The bar chart displays the Success Rate by Call Duration in various time intervals. Interestingly, the shortest calls (0-1 minute) have the highest success rate, and there's a noticeable drop as calls lengthen to 1-2 minutes. As the call duration increases beyond two minutes, the success rate seems to stabilize, with minimal variations across the longer time intervals.

This pattern could indicate that the most straightforward issues or easiest payments to collect are resolved quickly, hence the high success rate for very short calls. The leveling off of the success rate after the initial drop suggests that once a call extends past a certain point, the likelihood of success does not vary much with additional time. This insight could be pivotal for optimizing call center operations, possibly indicating an optimal call length for successful outcomes.

Part 2: Call Outcome Model Preparations

Feature Engineering:
New features were constructed from existing features and outside sources

New Features

Write-Off Score:

•A new write-off score variable was constructed from the account data.

•It is the probability of an account being a write-off.

•Multiple classification models were used, including LG, XGBoost, and NN.

• The XGBoost model was chosen as it has the most accurate prediction.

Write-Off Score XGBoost Model

Model Preparations:

•The account dataset was filtered for accounts that have write-off or paid closed status.

•Categorical variables were dummified.

•All non-binary variables were normalized.

Dealing with missing values:

•Charge-off Amount (Coamt), Income Type 2 (IncType2), and Promotion (Promo) variables were removed as they have 83%, 70%, and 67% missing values respectively. 

•Rows, where Income Type 1 (IncType1) was missing, were dropped, this only reduces the data size by about 1%.

No data imputations were needed.

•The final model data has 20,408 rows.

•The data was split into 70% training and 30% test.

Model Results on Test Data:

The original account dataset was filtered the same way the model data was, and then the XGBoost model was applied to it to predict the Write-off Score probability for all rows.

Picture1.png

Additional Features

Economic Indicators

In the predictive modeling project focused on assessing the likelihood of successful call outcomes for loan payments, key economic indicators have been included to complement individual data. The Consumer Price Index (CPI) and the Bank Prime Loan Rate (DPRIME) were selected for their relevance in reflecting the overall economic health and its impact on consumer financial behavior. Incorporating these indicators enhances the predictive accuracy of the model by providing a broader context of economic conditions.

Consumer Price Index (CPI)

A measure of the average change in prices over time for a market basket of consumer goods and services.

Data Source: Federal Reserve Bank - URL: https://fred.stlouisfed.org/series/CORESTICKM159SFRBATL

Bank Prime Loan Rate (DPRIME)

The interest rate that banks charge their most creditworthy customers and is the starting point for many other loan rates.

Data Source: Federal Reserve Bank - URL: https://fred.stlouisfed.org/series/DPRIME

Additional Drived Features

Call_Outcome

Feature Definition: Whether the outcome of a call was a success or failure.

Delinq_Duration

Feature Definition: Measures the time between the first and last missed payments on an account, in months .

It is calculated by comparing the payment dates and due dates in each account's payment history.

Theoretical Rationale: This feature distinguishes accounts based on the duration of their payment issues. For example, an account with missed payments only over a short period likely faced a temporary finanial crisis, indicating a short Delinq_Duration. Conversely, missed payments spread over several years suggest ongoing financial difficulties, leading to a longer Delinq_Duration.

Implication for Predictive Accuracy: Delinq_Duration provides insight into the persistence of payment problems, helping to predict future payment reliability more effectively.

CallCount

Feature Definition: CallCount represents the total number of calls made to an account.

Theoretical Rationale: The purpose of this feature is to help the model differentiate between accounts based on their response to outreach efforts. For example, imagine an account that has four successful resolutions from ten calls made in response to past delinquencies. This indicates that while the account does eventually respond, multiple calls are often necessary to achieve a resolution. In contrast, another account might also show four successful resolutions but was contacted only four times. This suggests a higher rate of responsiveness and efficiency, as each call effectively led to a resolution without the need for repeated contact.

Implication for Predictive Accuracy: By evaluating the total number of calls against the number of successful resolutions, CallCount provides critical insights into the effectiveness and efficiency of account responses. This distinction is key for forecasting the effort and resources needed for future interactions, optimizing contact strategies, and enhancing the overall management of account delinquencies.

Aggregated PmtAmt, PrinAmt, IntrAmt

Feature Definitions: These variables represent the total payment amount, total principal amount, and total interest amount, respectively, across all transactions for an account.

Theoretical Rationale: Aggregating these payment metrics allows the model to have a comprehensive view of an account's payment history up to the time of the call. This holistic approach ensures that the model considers the entire financial behavior of the account, rather than isolated transactions. For example, knowing the total principal or total payments made can provide insights into how much of the loan has been paid off, which is crucial for assessing the current financial standing and commitment of the borrower.

Implication for Predictive Accuracy: These aggregated values help the predictive model more accurately gauge an account’s financial health and repayment capability at any given point. They reflect the overall engagement of the borrower with the loan terms over time and are instrumental in forecasting future payment behaviors and potential delinquencies. By integrating the full payment history, the model can make more informed decisions, enhancing its ability to predict outcomes of calls and manage credit risks effectively.

 

PmtAmt_per, PrinAmt_per, IntrAmt_per

Feature Definitions: These variables represent the payment amount, principal amount, and interest amount, respectively, as percentages of the loan amount for each account.

Theoretical Rationale: These features are designed to normalize payment information across different loan sizes. By converting absolute amounts into percentages, the model can compare payment behaviors on a consistent scale. This is especially useful because accounts with larger loan amounts naturally tend to have larger nominal payments, which could skew the model's assessment if only absolute amounts were considered. For instance, a $1,000 payment on a $10,000 loan (10%) is quite different from the same $1,000 on a $100,000 loan (1%). By using percentages, these features allow the model to accurately gauge the relative financial commitment an account has made towards settling their loan, regardless of the loan size.

Implication for Predictive Accuracy: Utilizing these percentage-based metrics allows the predictive model to fairly assess and compare the payment performance of all accounts, irrespective of the underlying loan amounts. This standardization is crucial for identifying trends and patterns in payment behavior that are not merely artifacts of loan size but are indicative of the borrower's willingness and ability to pay.

PmtDuration

Feature Definition: PmtDuration is the number of months between the first and last payments made on an account.

Theoretical Rationale: This feature provides insight into the duration over which an account has been actively making payments. Measuring the span from the first to the last payment helps the model understand the longevity of the payment behavior. For example, an account that continues making payments over a long period demonstrates sustained financial engagement and commitment, whereas an account with a shorter PmtDuration may indicate quicker loan repayment or an early cessation of payments, each of which has different implications for financial stability and risk assessment.

Implication for Predictive Accuracy: By assessing the length of the payment period, PmtDuration aids in distinguishing between long-term and short-term borrower engagement. This duration metric is crucial for evaluating the consistency and reliability of an account’s payment history. It helps the model predict future payment behaviors more accurately by considering the historical length of payment activity, thus optimizing loan management strategies and risk assessment practices.

Call Outcome Model Building

Multiple classification models were used, and a neural network was chosen as it performed consistently better than all other models.

Model Preparations

•The Account, Call, and Payment datasets were combined.

•Categorical variables were dummified.

•All non-binary variables were normalized.

Dealing with missing values:

•Charge-off Amount (Coamt), Income Type 2 (IncType2), and Promotion (Promo) variables were removed as they have 91%, 89%, and 63% missing values respectively. 

•Rows, where Income Type 1 (IncType1) was missing, were dropped, this only reduces the data size by 0.15%.

No data imputations were needed.

•The final model data has 34,225 rows.

•The data was split into 80% training and 20% test.

•The NN has 6 layers.

•Specificity is the true negative rate. 

Picture2.png
Picture3.png
Picture4.png
Picture1.png

With and Without The New Features

Picture6.png
Picture7.png
Picture8.png

The inclusion of new features in the predictive model has evidently enabled the identification of more nuanced patterns within the data. This enhancement in pattern recognition is crucial, as it suggests a greater ability of the model to utilize the full spectrum of information provided, thereby improving the accuracy of its predictions.

Comparing the two confusion matrices, the stark reduction in False Positives with the new features cannot be overlooked. This improvement is critical in our context, where precision is highly valued. The neural network model, specifically chosen for this task, has outperformed other models like XGBoost, which recorded 140 False Positives, and logistic regression, which had 160. The lower False Positive rate achieved by the neural network represents a significant advance, demonstrating the model's refined capability to distinguish between actual positive outcomes and those that are not, thus underpinning the reliability of the predictive process.

The Receiver Operating Characteristic (ROC) curve illustrates the performance of a call success prediction model. The curve hugs the top left corner, showing a high True Positive Rate (sensitivity) and low False Positive Rate (for nearly all thresholds, which is a sign of good model performance. The area under the curve (AUC) is 0.93, which is very close to 1, indicating a high overall ability of the model to distinguish between successful and unsuccessful calls. A perfect model would have an AUC of 1, so an AUC of 0.93 suggests the model is making accurate predictions with a low rate of false positives.

Picture9.png

The Precision-Recall curve displayed here represents the trade-off between precision (the proportion of true positives among all identified positives) and recall (the proportion of true positives identified among all actual positives) for a call success prediction model.

The curve stays high across the recall range, which means the model maintains a high precision even as it successfully captures a large proportion of positive cases. An area under the curve (AUC) of 0.99 is outstanding, indicating that the model has a high precision across all levels of recall. This is an excellent result, suggesting that the model is very good at predicting call success, yielding very few false positives (calls predicted as successful that were not) and capturing most of the true successes.

Account Journey
How the model should be implemented

Screenshot 2024-04-07 121532.png

This graph serves as an illustrative example of how the Call Success model might be applied over the life of an account. It emphasizes the impact of time-changing variables on the model's predictions, focusing here on how a single dynamic variable, such as missing payments for over 10 days, can significantly affect the likelihood of successful collections.

Although other variables naturally evolve with time and can influence outcomes, in this illustration, all factors remain constant except for 'Delinq10' and 'Total Delinquency.' This deliberate simplification helps to isolate and demonstrate the substantial effect that increasing delinquency has on the model's forecast for successful payment recovery.

The account, despite active contact with the collections department, shows a diminishing probability of making payments as delinquency grows. The high 'WriteOff_Score' already suggests a considerable risk. As the 'Delinq10' variable increases, this is starkly reflected in the model's decreasing prediction probabilities, underscoring how critical timely payments are in the context of debt collection and risk management.

bottom of page