Wasting Money with Data Science

Slides of the presentation I gave at the Amsterdam Dataiku meetup on 01/05/2019

Transcript

1. Wasting Money with DS Giovanni Lanzani — Director of L&D @gglanzani on Twitter
2. WHO AM I
3. WHO ARE WE
4. Our clients 4
5. HOW DID WE GET THERE?
6. HYPE 6 (And FOMO)!
7. What people got 7 • A lot of POCs • A lot of screenshots / presentations / dashboards on a laptop • Nice stories to tell to their network, about those screenshots and especially those dashboards • Headaches with data and infra even more scattered
8. But… 8 • We got a data scientist working on trees, and forests • Neural Networks! • Deep Learning!!!
9. What do companies actually need 9 • Put things of value into production
10. The three hurdles Credit to Jon Shave gdd.li/lavaredo
11. Oversimplifying 11 Requirements Data Sources Exploration Modeling Products Feedback Data scientist ML engineer Data engineer Data engineer !" Customers
12. Kaggle Curse 12 • gdd.li/toldYouSo • Many data scientists approach the problem at hand with a Kaggle-like mentality: delivering the best model in absolute terms, no matter what the practical implications are. • In reality it's not the best model that we implement, but the one that combines quality and practicality: a continuous balancing act • Netflix competition
13. Solving THEM
14. Data 15 Data {insert something here} should be pro grade
15. Skills 16 • Participate in actually building production quality systems OR being proficient enough in R or python to hack together a prototype on a very small dataset? • Supply of the second group keeps growing while demand is flat or shrinking • Especially as executives get burned by “data scientists” who don't know how to help them build things of value
16. HOW TO CREATE VALUE?
17. The value-chain of Data Science Action Value Prediction Data Model Optimize Measure “Knowledge becomes power through action!” 18
18. More than just a technical challenge Action Value Prediction Data Model Optimize Measure Business Tech Organizational challenges • Align with business processes • Accept and take action • Measure business value Technical challenges • Collect and store the data • Build a predictive model • Optimize and make actionable 19
19. Start at the value and work your way back Action Value Prediction Data Model Optimize Measure Value Proposition End User Business objectives 20
20. Finding the fit between the end-user and the analytics value proposition • What are the business goals you are (currently) aiming for and what are relevant KPI’s? For example, increase of net profit of traditional trade outlets. • What predictions and optimizations can help for achieving these business goals? For example, the prediction of potential outlet profitability. • Who is the end-user that takes actions based on the predictions and optimizations? For example, a sales manager can allocate coolers by using predicted outlet profitability. • What relevant data is available for building the advanced analytics powered product? Ideally very detailed data, delivered as frequently as possible, and with an available historical set of 4-5 year. Can be both internal and external data. 21
21. Example: Business objective Predicting probability of churn of high-value customers that will optimize overall marketing spend by identifying critical customers that are likely to churn and more efficiently allocating spend 22
22. Before you start the project • Understand the business value for the stakeholders • Know the end-user you need to empower in creating value • Develop a good value proposition for the end-user 23 Value Proposition End User Business objectives
23. Aim for developing a data MVP 24 Delightful Usable Reliable Functional Model Pipeline Data Delightful Usable Reliable Functional Model Pipeline Data Model Not this… ...but this!
24. 6-8 weeks to go from idea to data MVP 25 Data & Pipelines Insight Visualizations Predictive Model Data MVP Business Values Data Sources E(T)L EDA Modeling Product- ionize Evaluate Sprint 1 Sprint 2 Sprint 3 PoV
25. Starting up your project 26 • Always have a kick-off meeting with all stakeholders • Plan stand-ups, sprint-reviews and data delivery a.s.a.p. • Engage stakeholders through frequent demos • Scope you project and manage expectations: • Start small and fail fast • Aim for a Minimum Viable Product • Have a clear ‘Definition of Done’ • Plan to test your model in practice a.s.a.p. and measure the value
26. About getting the data 27 • Make sure start collecting the data a.s.a.p. • Get a S.P.O.C. for all the important data owners • Check data quality as soon as you receive data • Make sure your data is GDPR compliant (if in doubt ask) • Remember: filtering out a column is easier them requesting one! • Finally, led the business tell what you should do; the data will tell you if and how you can do!
27. Workflow characteristics 28 • Get to a working (simple) model as quickly as possible • Be agile; short cycles, incremental improvements and result driven • Iterative increments: • mining extra data sources • extension of the input datasets • continued feature and/or target engineering • optimization of the predictive model • Frequent demos of the ‘improved’ model to stakeholders
28. ANALYTICS TRANSLATORS?
29. The need for analytics translators 30 • The link between the business case and the data teams is crucial
30. The role of analytics translators 31 And how can you help! • Ensures that business challenge is addressed • Translates between business and analytics • Helps identify use cases • Drives implementation of solutions • Advocates on data driven decision making
31. Giovanni Lanzani @gglanzani on Twitter https://gdd.li/wasting