LINE Score: How to build Alternative Credit Scoring Model

Sangwoo Kim
LINE Plus
https://linedevday.linecorp.com/jp/2019/sessions/B2-3

Transcript

1. 2019 DevDay LINE Score: How To Build Alternative Credit Scoring Model > Sangwoo Kim > LINE Plus
2. > Sangwoo Kim > Manager of Credit Scoring & Risk Management team > Responsible for developing LINE Score This session will benefit those who are interested in data analytics, especially credit score modeling Hello Developers and Data Scientists!
3. > Developing a scoring model > Credit scoring system Credit scoring model Building a scoring model
4. Agenda > Key Concept > LINE Score as an Alternative Credit Score > Overview of Credit Score > Techniques for Credit Score Modeling > Measurement methodology for performance > Achievements of LINE Score
5. Key Concept
6. Predictive Model vs. Credit Scoring Model Predictive Model The general predictive model usually used in non-financial companies to achieve a specific purpose. > Mostly focusing on efficiency or effectiveness > Indirectly affects businesses or companies Credit Scoring Model The credit scoring model usually used in financial institutions or credit bureaus to evaluate a customer’s creditworthiness. > Mostly focusing on default > Influenced by various regulations > Directly affects the profit or losses of financial institutions
7. Credit Score vs. Alternative Credit Score Credit Scoring Model The credit scoring model is a traditional evaluation method that usually uses financial data. > Uses financial data for scoring > Mainly used in countries with advanced financial industries > Only for those who have a history of financial transactions in the past Alternative Credit Scoring Model The alternative credit scoring model is an evaluation method that is in the limelight recently. It usually uses non-financial data. > Uses non-financial data for scoring > Available in all countries > Possible for those who don't have a history of financial transactions in the past
8. LINE Score
9. “Enrich daily life” LINE Score
10. Evaluate credit score by analyzing behavior data on the LINE platform LINE Score will only be calculated upon the user's consent
11. Various services are running on the LINE platform Big Data
12. An ecosystem in which data continuously flows into LINE scores can be utilized in various businesses at the same time Creating the LINE Score Platform Data source Financial Institutions Sharing Economy C2C Platforms Reservation Services Rental Businesses Score platform
13. Overview of Credit Scoring
14. > A credit scoring model for evaluating customers who apply for loans Application Scoring System Behavior Scoring System > A credit scoring model for evaluating the behavior of existing loan customers Credit Scoring System > A statistical model that utilizes a customer's past history to predict creditworthiness in the future Key Terminology
15. > Descriptive information used to predict defaults Feature or Variable (Input) ‘Build’ or ‘Develop’ credit scoring model > A process of developing a statistical model that predicts 'defaults' using 'features' Default (Target) > Mainly uses a loan delinquency history of more than 60 days or 90 days as target information Key Terminology
16. Basic Concept of Credit Scoring Model How to Build a Credit Scoring Model Past default record (y) Past transaction history (x) Fitting the best algorithm to predict default y = f (x) How to Operate a Credit Scoring Model Funtion Loan applicant's future possibility of default Loan applicant's past transaction history Selected features (x) Probability of default (y)
17. Techniques for Credit Score Modeling
18. Process Overview The credit scoring model building process can be divided into 5 steps as follows: Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition
19. Process Overview The credit scoring model building process can be divided into 5 steps as follows: Establishing the purpose of the model and selecting groups or subjects to be used for model development Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition
20. Process Overview The credit scoring model building process can be divided into 5 steps as follows: Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition Selecting the target value that needs to be predicted
21. Process Overview The credit scoring model building process can be divided into 5 steps as follows: Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition Creating and selecting features that are highly predictable and highly correlated with the target
22. Process Overview The credit scoring model building process can be divided into 5 steps as follows: Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition Exploring the highly predictable and optimal algorithms suitable for the purpose of model development
23. Process Overview The credit scoring model building process can be divided into 5 steps as follows: Feature creation & selection Model fitting Scaling & calibration Purpose of model Target definition Converting to a score or grade form that is easy for customers to understand
24. Several types of credit scoring models for different purposes > Behavior Scoring Model > Collection Scoring Model > Application Scoring Model Purpose of Models > Fraud Detection Model > Early Warning Model
25. Set up a target that fits the model’s purpose Target Definition > Behavior Scoring Model > Collection Scoring Model > Application Scoring Model > Fraud Detection Model > Early Warning Model → Event of default → Event of default → Possibility of collection → Event of fraud → Event of issue related to credit risk
26. Feature Creation and Selection Examples of candidate feature creation > Type of mobile phone > Number of received or sent calls in a day/week/month/year > The ratio between sending and receiving calls/messages > Main time zone
27. Factors to consider when selecting features > Information value > Stability > Diversity > Correlation Feature Creation and Selection
28. Traditionally, logistic regression has been widely used Recently, machine learning has also been used to increase predictability Model Fitting Logistic Regression A model usually can be derived in the form of a highly-explainable scorecard > Higher Interpretability > Simple > Relatively small amounts of data > Relatively low performance Machine Learning Algorithm A model usually can be derived in the form of a highly-complex and hard-to- interpret black box > Lower interpretability > Complicated > Requires a large amount of data > Relatively high performance
29. Process for improving user utilization and understanding Scaling and Calibration Scaling Calibration Probability of Default Score 0.000% ~ 0.124% 1,000 ~ 901 0.125% ~ 0.249% 900 ~ 876 0.250% ~ 0.374% 875 ~ 851 0.375% ~ 0.499% 850 ~ 826 0.500% ~ 0.624% 825 ~ 801 0.625% ~ 0.749% 800 ~ 776 0.750% ~ 0.874% 775 ~ 751 … … Score Range Grade 820 ~ 1000 1 800 ~ 819 2 780 ~ 799 3 750 ~ 779 4 730 ~ 749 5 700 ~ 729 6 650 ~ 699 7 … …
30. Measurement Methodology for Performance
31. Measuring the performance of a model in terms of stability and discriminability How to Measure Model Performance Stability Indicator The difference in the distribution between the development data set and validation data set, which is used to verify the stability of the features, score, and grade > PSI Discriminative Power A numerical measure of how well rank ordering was done on predicting the target, which is used to show the discriminant power of the features, score, and grade > AUROC (AR) > K-S
32. Discriminant Power Indicator: AUROC False Positive Rate True Positive Rate Perfect Model Developed Model Random Model Area Under the ROC = A+B A B
33. Assume that there are 5 customers in default out of 100 customers, and 5 customers in default are of the 5th, 10th, 20th, 25th, and 30th lowest scores Example of AUROC Calculation False Positive Rate True Positive Rate Random Model A
34. Assume that there are 5 customers in default out of 100 customers, and 5 customers in default are of the 5th, 10th, 20th, 25th, and 30th lowest scores Example of AUROC Calculation False Positive Rate True Positive Rate Developed Model Random Model A Perfect Model 5 10 20 25 30
35. Assume that there are 5 customers in default out of 100 customers, and 5 customers in default are of the 5th, 10th, 20th, 25th, and 30th lowest scores Example of AUROC Calculation False Positive Rate True Positive Rate Developed Model Area Under the ROC = A+B Random Model A Perfect Model 5 10 20 25 30 B
36. Achievements of LINE Score
37. Achievements of LINE Score Offer Save Increase many benefits based on LINE Score costs by providing more reasonable financial services accessibility to financial services
38. Thank you