Andy Young – Data-Driven Growth: Analytics Tools and Tips for Marketers in 2016 (Turing Fest 2016)

Transcript

1. Analytics tools and tips for marketers in 2016 Data-driven Growth 📈 🚀 Andy Young, 500 Startups / Freelance Turing Festival 2016
2. Hi, I’m Andy! @andyy #fb Photo by Dan Taylor - dan@heisenbergmedia.com
3. Agenda Why analytics? Deconstructing the modern analytics stack Lies, damn lies, and tech metrics..
 - analytics in the real world
4. 🤔 Why analytics?
5. Don’t start with the data
6. Don’t start with the data
7. Common Analytics FAILS • Drowning in data • Not identifying 
 key questions to answer • Not starting with
 clear hypotheses
8. We need to know how we’re doing
9. • Business/product overall • Individual experiments, campaigns, channels, customer segments We need to know how we’re doing
10. We need to know how we’re doing • Business/product overall • Individual experiments, campaigns, channels, customer segments • What is working? • What is not? • Where to focus for improvement?
11. Analytics Pros.. • Start with a hypothesis • Identify, collect + analyse necessary relevant data • Conclude & action • Iterate, revise
12. Analytics for Growth 📈
13. Growth? •More users •More revenue
14. Analytics techniques • Early stage:
 - innovation accounting 
 (Lean Startup)
 - cohort analysis
15. Analytics techniques • Early stage:
 - innovation accounting 
 (Lean Startup)
 - cohort analysis • Growth stage:
 - growth accounting
 - customer segmentation
 - channel analysis
 - funnel economics
16. Early stage questions
17. Are we getting traction?
18. “Traction is the rate at which monetizable value is extracted from customers”
19. Innovation accounting
20. Cohort analysis?
21. Growth stage questions
22. Are we growing?
23. Growth Accounting
24. 👌 Growth Accounting
25. Funnel performance?
26. Growth stage • Which channels drive the most customers? • Which channels drive the best customers?
27. How much do we want to pay per click?
28. How much do we want to pay per click? ..as much as possible?! Outspend competitors Activate new channels
29. Customer segmentation & channel performance For different groups of customers: • Where do they come from? • How do they behave?
30. Andy Young // @andyy // andy@500.co Customer segmentation Users acquired via different channels will have different behaviours Different cohorts will have different experiences of your product Different users will have been exposed to different A/B tests
31. Andy Young // @andyy // andy@500.co These are all properties of your users UTM tags: source, medium, campaign, terms Landing page Signup time A/B test buckets Referrer Viral source Customer segmentation
32. Demystifying direct traffic
33. Tag all the things! Direct + Referral traffic => specific source/medium. 
 Email, Social, Offline Tagging guide Real-world examples blog post
34. Brand vs. non-brand terms for search channels (both organic + paid)
35. Landing page:
 often a big clue to source!
36. Attribution models • First touch? • Last touch? • Multi-touch? Contributing channels?
37. Cross-device? Offline channels?
38. Cross-device • Common: Research on mobile/tablet, purchase on tablet/desktop • Challenge: attribution on mobile • Need: unique identifying information • E.g. email, mobile/cell #
39. Cross-device attribution
40. Filling the attribution blanks..
41. Geographic A/B tests
42. Most campaigns FAIL
43. Most campaigns fail • ..but most every failed campaign will contain partial successes • Plan in advance: how will we know
 at which point in the user journey a campaign failed? • Do things that don’t scale
44. Pulling this all together Why analytics? Deconstructing the modern analytics stack Lies, damn lies, and tech metrics..
 - analytics in the real world
45. What to track? • Discovery / Acquisition • Activation • Engagement • Conversion / Purchase • Retention • Referral
46. So many platforms
47. So many platforms ¯\_(ϑ)_/¯
48. What functionality do you need? • Session/Pageview Analytics
 e.g. Google Analytics, Chartbeat • User/Event-based analytics
 e.g. Kissmetrics, Mixpanel/Amplitude, Localytics • Mobile deep linking and attribution
 e.g. Branch, Adjust, AppsFlyer, Tune, Kochava • Mobile-specific
 e.g. Flurry, Swrve, Leanplum • A/B testing
 Optimizely, WVO, Google Content Experiments, Mixpanel
49. What functionality do you need? • Session/Pageview Analytics
 e.g. Google Analytics, Chartbeat • User/Event-based analytics
 e.g. Kissmetrics, Mixpanel/Amplitude, Localytics • Mobile deep linking and attribution
 e.g. Branch, Adjust, AppsFlyer, Tune, Kochava • Mobile-specific
 e.g. Flurry, Swrve, Leanplum • A/B testing
 Optimizely, WVO, Google Content Experiments, Mixpanel • Querying & Charting
 e.g. Mode, Periscope, Tableau, RJMetrics • Dashboards
 e.g. Geckoboard, Klipfolio • Audience demographics, interests & rankings
 e.g. Quantcast, Comscore, Alexa, SimilarWeb • Marketing Automation, CRM, Email & Push Notifications
 e.g. Marketo, Hubspot, AppBoy, Kahuna • Platforms
 e.g. Google Firebase, AWS Mobile Analytics
50. What functionality do you need? • Session/Pageview Analytics
 e.g. Google Analytics, Chartbeat • User/Event-based analytics
 e.g. Kissmetrics, Mixpanel/Amplitude, Localytics • Mobile deep linking and attribution
 e.g. Branch, Adjust, AppsFlyer, Tune, Kochava • Mobile-specific
 e.g. Flurry, Swrve, Leanplum • A/B testing
 Optimizely, WVO, Google Content Experiments, Mixpanel • Querying & Charting
 e.g. Mode, Periscope, Tableau, RJMetrics • Dashboards
 e.g. Geckoboard, Klipfolio • Audience demographics, interests & rankings
 e.g. Quantcast, Comscore, Alexa, SimilarWeb • Marketing Automation, CRM, Email & Push Notifications
 e.g. Marketo, Hubspot, AppBoy, Kahuna • Platforms
 e.g. Google Firebase, AWS Mobile Analytics 😱
51. How to pick? • What functionality do you need? • Which platforms? Web, mobile, server? • Which other tools/data do you need to integrate with? • Who will be using it? Devs, data scientists, product folks, marketing people? • How do you want to use it? Analysis, reporting, dashboards.. • What data volumes? Data points, events per second, data points per month. • Budget?
52. How to pick? • What functionality do you need? • Which platforms? Web, mobile, server? • Which other tools/data do you need to integrate with? • Who will be using it? Devs, data scientists, product folks, marketing people? • How do you want to use it? Analysis, reporting, dashboards.. • What data volumes? Data points, events per second, data points per month. • Budget?
53. How to pick? Things to be wary of: • Data lock-in, future portability • Building your own
54. Tools are not a panacea
55. Biggest single factor?
56. Biggest single factor? The quality of your analytics is mostly driven by the quality of your tracking implementation: • Coverage/depth • Accuracy/lack of bugs
57. Use a tag manager
58. Data collection: Segment.com
59. Andy Young // @andyy // andy@500.co www.sitehound.co Free code!
60. SQL?
61. Andy Young // @andyy // andy@500.co Use your existing database Users Learn SQL! It's not hard Just need a slave database for analytics - “read replica” - i.e. a live copy
62. Andy Young // @andyy // andy@500.co Use your existing data Users SELECT COUNT(*) FROM users
63. Andy Young // @andyy // andy@500.co Use your existing data Users SELECT COUNT(*) FROM users WHERE created > ‘2013-07-01’ AND created < ‘2013-08-01’
64. Andy Young // @andyy // andy@500.co Use your existing data SELECT COUNT(*) FROM users LEFT JOIN sales USING (user_id) WHERE users.created > ‘2013-07-01’ AND users.created < ‘2013-08-01’ AND sales.date < DATE_ADD(users.created, 1 MONTH)
65. Dashboards?
66. What makes a good dashboard?
68. Growth model spreadsheet • Annotate with notes on definitions • Review weekly - zoom in vs. zoom out • Share with whole team • Use to predict and prioritise • What to look at weekly vs. monthly/quarterly
69. One Metric That Matters
70. Metrics are people too Measure every metric in terms of unique people or % conversion
71. Joining the dots Plugging all these tools and platforms together
72. Initially, manual is OK!
73. Initially, manual is OK! • Forcing function to ensure we: • learn what works • understand the data • need the data • Paste weekly into a Google Sheet/Excel
74. Automate over time to reduce effort
75. Plumbing for automation • Zapier, Tray.io • Supermetrics, Blockspring for Google Sheets/Excel • Segment.com • Own database/data warehousing - AWS RDS/Redshift • Custom queries - Tableau, Periscope etc
76. The numbers never add up Why analytics? Deconstructing the modern analytics stack Lies, damn lies, and tech metrics..
 - analytics in the real world
77. The 5 stages of (Analytics) Grief..
78. The 5 stages of Analytics Grief • Denial
79. FB ad clicks vs. GA visits
80. The 5 stages of Analytics Grief • Denial • Anger
81. Mixpanel vs. own database
82. The 5 stages of Analytics Grief • Denial • Anger • Bargaining
83. The 5 stages of Analytics Grief • Denial • Anger • Bargaining • Depression
84. Where did it all go wrong?
85. Where did it all go wrong? • The data is bad • Our definitions are wrong
86. The numbers never add up • Absolute truths • e.g. Signups, Transactional data, UGC, data from our own DB • Lossy/noisy measurements • e.g. Client-side tracking: GA, Mixpanel/Amplitude etc • Proxy metrics • e.g. … • Nuanced definitions • e.g. uniques vs. totals, funnel conversion rates
87. Even if the numbers do add up, they are still often misleading
89. Data discrepancies
90. Data discrepancies • GA vs. Mixpanel • Definition of unique user - cookie rules • Ad/tag blockers
91. Data discrepancies • Page did not finish loading - e.g. tag manager, or user gave up & hit back button • Cross-browser JS bugs? • Definition mismatches • Uniques vs totals • Funnels
92. Measure Twice: Compare data across different tools https://jeffpeachey.com/2010/12/02/measure-twice-cut-once-woodcut-for-sale/
93. The 5 stages of Analytics Grief • Denial • Anger • Bargaining • Depression • Acceptance
94. 3-5% discrepancies between platforms? Roll with it.
95. Large (>5%) difference? Debug/analyse to figure out why
96. Takeaways?
97. Secrets to successful analytics • Simplify • Be precise about what to measure, and why • Plan in advance • Data where you need it • Move to automate required work for weekly/monthly calculations • Document all analysis with simple bullet-points
98. The numbers tell us What They don’t tell us Why
99. Thanks, & good luck! @andyy andy@apexa.co.uk
100. References & further reading • My Analytics presentations • KPI sheet • SiteHound • UTM Tagging Guide • Lean Analytics • Social Capital - Accounting for User Growth