Predicting the effectiveness of churn prevention measures

Losing customers, also referred to as churning, is something that any company wants to prevent. In this talk, I will demonstrate how you can predict the unbiased effect of countermeasures, such as a discount or an additional service, on specific customers. Since we are interested in the causal relationship between countermeasure and churn, we need to go beyond analyzing correlations and use custom machine learning approaches. The presented approach will combine Deep Learning and Bayesian modelling in a custom setup.

Related blogpost: https://medium.com/bigdatarepublic/for-effective-treatment-of-churn-dont-predict-churn-58328967ec4f

Transcript

1. Predicting the Effectiveness of Churn Prevention Measures Gerben Oostra – Machine Learning Engineer @ BigData Republic
2. Churn is a revenue drain Churn prevention * 25% & 75% quantiles, depends on business size. https://www.profitwell.com/blog/average-revenue-churn-rate-benchmarks Companies lose between 2% and 16%* of revenue every month due to churn.
3. 3 issues within churn prevention Churn prevention 1: Models predicting churn 2: Assuming correlation is causation 3: Select the best predicted treatment
4. Running example: Telco (internet) Churn prevention Telco Inc Provides subscriptions with TV & Internet & Mobile Has a lot of diverse customers
5. Churn prevention Issue 1: Models predict churn Churn Classification Model Do nothing Direct mail Telemarketing Door 2 door Choose between Features Predicted churn propensity Policy Train Historical data
6. Action P(churn | A) P(retention | A) Uplift 0.6 0.4 0 @ 0.65 0.35 -0.05 TM 0.4 0.6 0.2 D2D 0.35 0.65 0.25 Classification Model Direct mail Telemarketing Door 2 door Features Predictions Solution 1: Predict uplift Policy Uplift(A,x) = P(retention | A, x) – P(retention | , x) Transformed Outcome Trick Regression model with labels (0, -2, 2) Athey, S., & Imbens, G. W. (2015). Machine learning methods for estimating heterogeneous causal effects. stat, 1050(5). Do nothing Uplift Regression Model The default
7. Churn prevention CLV € 300 Cost € 0.- € 0.25 € 5.- € 20.- Solution 1 : Base policy on economic result Action Uplift 0 @ -0.05 TM 0.20 D2D 0.25 × Result € 0.- € -15.25 € 55,- € 55,- − = Model Policy
8. Churn prevention Issue 2: Assuming correlation is causation T R Retention Treatment Features Causal graph Correlation has predictive power We need causation for prescriptive power We only observe correlations Here correlation is causation
9. Churn prevention Issue 2: Assuming correlation is causation T R Retention Treatment Features Historically: Past retention campaigns Future: Our model Causal graph Correlation has predictive power We need causation for prescriptive power We only observe correlations Correlation is not causation
10. Churn prevention Solution 2: Causal model (Inverse propensity weighting) T R Retained? Treatment Features Propensity model to learn the correlation Propensity Model Age / location / .. P(T | x) @ TM $ = 1 | Age / location / .. @ TM * + $ Propensity Model | → Weight samples inverse to propensity
11. Churn prevention Issue 3: Select the best predicted treatment Action Result € 0.- @ € -15.25 TM € 55,- D2D € 55,- Greedy Policy 100% Exploit limits feedback • We learn if selection worked • We never learn alternatives Feedback (Future) training data
12. Churn prevention Solution 3: Balance exploration & exploitation (Thompson sampling) Action Uplift @ -0.05 TM 0.2 D2D 0.25 Uplift P(Uplift | T) @ TM D2D Most likely Uplift Underlying distributions of Uplift Use sample as prediction Action Uplift Result @ 0.5 € 149.75 TM 0.1 € 25 D2D 0.2 € 40 Action Uplift Result @ -0.5 €-150.25 TM 0.3 € 85 D2D 0.15 € 25 Action Uplift Result @ -0.5 €-150.25 TM 0.3 € 85 D2D 0.15 € 25 Action Uplift Result @ -0.5 €-150.25 TM 0.3 € 85 D2D 0.15 € 25 Bayesian modelling
13. Churn prevention Preventing churn with 3 complementary techniques Model Policy (Re)Train 1: Actionable predictions: • Predict Uplift • Policy on business value 2: Causal model: • Inverse propensity weighting 3: Balance exploration & exploitation: • Thompson sampling • Bayesian modelling Predictions Action Feedback (churned/retained) Features Historical data Propensity Model Weights Transformed outcome Labels Features
14. Phone +31 (0)168 479294 Email info@bigdatarepublic.nl Coltbaan 4C, 3439 NG Nieuwegein, The Netherlands Address Questions?