Survival Analysis - Unbounce Use Case

Transcript

1. Ha Dinh, Data Analyst at Unbounce Survivability Power Hour February 5th Data Science, Winter 2020
2. | Copyright 2018. All rights reserved. About Ha Bachelor, Marketing & Business Econ @ Uni. Cincinnati Master of Data Science @ UBC Co-organizer, Women in Data Science Vancouver Area of interest: customer behavior
3. | Copyright 2018. All rights reserved.
4. | Copyright 2018. All rights reserved. About Unbounce Landing page builder ~15k customers Canada’s Most Admired Corporate Cultures 2019 Reached gender parity in 2019 15 people in Data + 3 Data Dogs
5. | Copyright 2018. All rights reserved.
6. | Copyright 2018. All rights reserved. Goals Overview of Survival Analysis Survival Analysis Applications Unbounce Case study Beyond Survival Analysis
7. Overview of Survival Analysis
8. Overview of Survival Analysis Survival Analysis is used to estimate the lifespan of a particular population under study
9. | Copyright 2018. All rights reserved. A B C D E 1 month 3 months 2 months MEDICAL PATIENTS
10. 10 | Copyright 2018. All rights reserved. Overview of Survival Analysis Questions of Interest 1. What proportion of patients will survive within 3 months?
11. | Copyright 2018. All rights reserved. A B C D E 1 month 3 months 2 months MEDICAL PATIENTS
12. | Copyright 2018. All rights reserved. A B C D E 1 month 3 months 2 months 40% MEDICAL PATIENTS
13. 13 | Copyright 2018. All rights reserved. Overview of Survival Analysis Questions of Interest 1. What proportion of patients will survive within 3 months? 2. What proportion of patients will not survive within 3 months?
14. | Copyright 2018. All rights reserved. A B C D E 1 month 3 months 2 months MEDICAL PATIENTS
15. | Copyright 2018. All rights reserved. A B C D E 1 month 3 months 2 months 60% MEDICAL PATIENTS
16. 16 | Copyright 2018. All rights reserved. Overview of Survival Analysis Questions of Interest 1. What proportion of patients will survive within 3 months? 2. What proportion of patients will not survive within 3 months? 3. What is the risk of not surviving at 1 month, among those who have survived until that point?
17. | Copyright 2018. All rights reserved. A B C D E 1 month 3 months 2 months MEDICAL PATIENTS
18. | Copyright 2018. All rights reserved. A B C D E 1 month 3 months 2 months 0 MEDICAL PATIENTS
19. 19 | Copyright 2018. All rights reserved. Overview of Survival Analysis Important Components 1. Event of interest (churn, death, etc.) 2. Time 3. Censoring ◦ http://www.stats.ox.ac.uk/~mlunn/lecturenotes1.pdf
20. | Copyright 2018. All rights reserved. A B C D E 1 month 3 months 2 months MEDICAL PATIENTS
21. 21 | Copyright 2018. All rights reserved. Overview of Survival Analysis Survival Function Probability that the event of interest has not occurred at time t
22. 22 | Copyright 2018. All rights reserved. Overview of Survival Analysis Questions of Interest 1. What proportion of patients will survive within 3 months? 2. What proportion of patients will not survive within 3 months? 3. What is the risk of not surviving at 1 month, among those who have survived until that point?
23. 23 | Copyright 2018. All rights reserved. Overview of Survival Analysis Hazard Function Probability that the subject will experience an event of interest within a small time interval, provided that the individual has survived until the beginning of that interval
24. 24 | Copyright 2018. All rights reserved. Overview of Survival Analysis Questions of Interest 1. What proportion of patients will survive within 3 months? 2. What proportion of patients will not survive within 3 months? 3. What is the risk of not surviving at 1 month, among those who have survived until that point?
25. Survival Analysis Applications
26. 26 | Copyright 2018. All rights reserved. Survival Analysis Applications Life Insurance
27. 27 | Copyright 2018. All rights reserved. Survival Analysis Applications Mechanical Maintenance
28. 28 | Copyright 2018. All rights reserved. Survival Analysis Applications Employee Attrition
29. 29 | Copyright 2018. All rights reserved. Survival Analysis Applications Customer Retention
30. Unbounce Case Study
31. 31 | Copyright 2018. All rights reserved. Goal Validate whether a potential metric can be used as a retention metric. Data Science Approach • Kaplan Meier survival function Unbounce Case Study Identify Customer Retention Metric
32. 32 | Copyright 2018. All rights reserved. Unbounce Case Study Problem
33. | Copyright 2018. All rights reserved.
34. 34 | Copyright 2018. All rights reserved. Unbounce Case Study Customer Flow Sign up for 14-day trial January 02 Paid 1X January 16 Churn September 01 Paid 3X March 16 Login Add a new variant for A/B test Connect custom domain Make changes and republish page Invite an additional user
35. 35 | Copyright 2018. All rights reserved. Unbounce Case Study Value Proposition
36. 36 | Copyright 2018. All rights reserved. Unbounce Case Study Value Proposition “With Unbounce, you can create and optimize dedicated landing pages that prompt your visitors with one focused goal instead of leaving them to wander a site full of distractions”
37. 37 | Copyright 2018. All rights reserved. Unbounce Case Study Value Proposition “With Unbounce, you can create and optimize dedicated landing pages that prompt your visitors with one focused goal instead of leaving them to wander a site full of distractions”
38. 38 | Copyright 2018. All rights reserved. Unbounce Case Study Value Proposition
39. 39 | Copyright 2018. All rights reserved. Unbounce Case Study Finding Proxy Metric Optimize
40. 40 | Copyright 2018. All rights reserved. Unbounce Case Study Customer Flow Sign up for 14-day trial January 02 Paid 1X January 16 Churn September 01 Paid 3X March 16 Login Add a new variant for A/B test Connect custom domain Make changes and republish page Invite an additional user
41. 41 | Copyright 2018. All rights reserved. Unbounce Case Study Finding Proxy Metric Optimize Republish
42. Unbounce Case Study Should we put all effort in boosting page republishes?
43. 43 | Copyright 2018. All rights reserved. Overview of Survival Analysis Kaplan Meier Survival Analysis - Important Components 1. Event of interest: customer retention 2. Time: week (based on previous research) 3. Censoring: customers who still retain at the time of analysis -- we don’t know when will they stop paying to use Unbounce
44. 44 | Copyright 2018. All rights reserved. Overview of Survival Analysis Kaplan Meier Survival Analysis - Normalize Metric
45. 45 | Copyright 2018. All rights reserved. Overview of Survival Analysis Kaplan Meier Survival Analysis - Normalize Metric Normalize the data by using the proportion of action
46. 46 | Copyright 2018. All rights reserved. Unbounce Case Study Kaplan Meier Survival Analysis
47. 47 | Copyright 2018. All rights reserved. Unbounce Case Study Kaplan Meier Survival Analysis Weekly Republishes <> Retention
48. Beyond Survival Analysis Beyond Survival Analysis
49. 49 | Copyright 2018. All rights reserved. Beyond Survival Analysis Customer Profile Instruction on how to create the visualization above
50. 50 | Copyright 2018. All rights reserved. Beyond Survival Analysis Experiment (Split testing)
51. 51 | Copyright 2018. All rights reserved. Beyond Survival Analysis Data for Business Action
52. 52 | Copyright 2018. All rights reserved. Beyond Survival Analysis Data & Business
53. Thank You