Agile Data Science

Is Agile Data Science just two buzzwords put together? I argue that agile is a very practical and applicable methodology, that does work well in the real world for all sorts of Analytics and Data Science workflows. Here is what we've learned.

Transcript

1. 1 © All rights reserved to Agile Analytics in Mobile Gaming: lessons learned Volodymyr (Vlad) Kazantsev Head of Data Science at Product Madness 2015
2. 2 What we do?
3. 3 Heart of Vegas in charts iPad rankings, US iPad rankings, Australia
4. 4 Data Impact Team • Ad-hoc analytics and daily fires • Deep dive analysis; Predictive analytics • Data Engineering; R&D Team of 6
5. 5 Few Examples A B A/B Tests Customer Lifetime Value days $ value Segmentation group 1 group 2 group 3 group 4
6. 6 Technology Stack C++ ETL orchestration Transformation & Aggregation SQL Data Products Reports Dashboards +
7. 7 Lessons
8. 8 Lesson 1: Agile Philosophy for Data Science 1
9. 9 Agile Manifesto Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan * agilemanifesto.org
10. 10 Agile Data Science Manifesto Individuals and interactions over processes and tools Actionable insights over comprehensive reports Customer collaboration over project negotiation Responding to change over following a plan
11. 11 “If a building doesn’t encourage [collaboration], you’ll lose a lot of innovation and the magic that’s sparked by serendipity” - Steve Jobs Individuals and interactions over processes and tools
12. 12 Individuals and interactions over processes and tools Standing Desks + Easily Available Whiteboard
13. 13 Agile Principles Iterative, incremental and evolutionary Efficient and face-to-face communication Very short feedback loop and adaptation cycle Quality focus - iterations, timeboxed estimates - no to tasks by email (with no face-to-face) - daily standups, pair analysis - verifiable, reproducible findings
14. 14 Data Science Board
15. 15 Scrum-Ban in Data Science @ProductMadness • Weekly cycle • Daily standup meeting @10am • ToDo/WIP/Waiting buckets are kept small • Disruptions to weekly plan are expected • On-demand planning
16. 16 Lesson 1: Agile methods in Data Science 1. co-location matter; whiteboard next to your desk 2. Work with decision maker; share preliminary findings 3. Make a research plan; pivot early 4. Book “Findings” meeting before project start 5. MVP for Data Products 6. Do Daily Stand-ups !
17. 17 Lesson 2: Agile Velocity vs. Acceleration 2
18. 18 What is Agile Acceleration Waterfall Scrum Units of Work Time Interval Velocity = ΔVelocity = Acceleration* ΔTime VS.
19. 19 a = F m I run SQL, copy- paste data to Excel and send it by email I created a deep neural network to predict high spenders
20. 20 Case Study: to Git or not to Git Scripts (ruby, bash, python) Python Apps Python Modules IPython Notebooks Research Documents (word) Presentations (powerpoint) Spreadsheets (excel)
21. 21 Case Study: Git or not to Git Scripts (ruby, bash, python) Python Apps Python Modules IPython Notebooks ? Research Documents (word) Slides (powerpoint) Spreadsheets (excel)
22. 22 Case Study: Git or not to Git Scripts (ruby, bash, python) Python Apps Python Modules IPython Notebooks Research Documents (word) Slides (powerpoint) Spreadsheets (excel)
23. 23 Remove unnecessary weight
24. 24 Lesson 2: find the lightest suitable tool 1. IPython notebooks: Dropbox over Git 2. Google Slides over Powerpoint Google Slides over Email with images (>2 images) 3. Google Spreadsheets over Excel (for analytics) 4. Podio over Jira (for analytics) 5. Data Transformations in DWH in SQL over Hadoop 6. Don’t copy-paste code in IPython notebooks; use functions; don’t copy-paste functions in notebooks, use modules
25. 25 Lesson 3: Focus on Closing the Loop 3
26. 26 Analytics Loop Spot Opportunity Ask the Right Question Make Decision Improve the Business Data Science @work
27. 27 Analytics Spiral Ideas & Questions Data Analysis Insights Impact
28. 28 Data Science Value Pyramid Store & Query Reports Descriptive Analytics Predictive Analytics Data Products * inspired by Agile Data Science, Russell Jurney, O'Reilly Media 2013 Record what Happened Was it good or bad? Why did it happen? What will happen? Affect the outcome complexity value
29. 29 Data Science Value Loop Record what Happened Was it good or bad? Why did it happen? What will happen? Affect the outcome
30. 30 Limit the number of Open Loops 90% 90% 75% 80% 80% 60% 100% 100% 100% 100% 0% 0% Always prefer to have: 90% of tasks are 100% complete over 100% of tasks are 90% complete VS.
31. 31 Lesson 3: Focus on Closing the Loop 1. Don’t build predictive models that you can’t act upon. Don’t analyse stuff that does not help to make a decision 2. The best way to deal with Analytics Spiral is to avoid the spiral. Practise Crack a Case and “what if” method. 3. Climb the Data Value Pyramid fast. Once climbed - optimise the Data Value Loop. 4. Limit the number of “open loops”
32. 32 Lesson 4: Reproducibility Matters 4
33. 33 To the and back!
34. 34 Why? Boss: “Great! Can you run this for all monthly cohorts?” Because:
35. 35 Why? Because: Boss: “Sam is on holiday. Can you re-run his analysis?”
36. 36 Few IPython Tips
37. 37 Import all commonly used tools in one line. All access and security is abstracted away. Focus on SQL, not data access formatting and publishing a .png in one line of code PyCharm has great SQL editor
38. 38 Lesson 4: Reproducibility • Get rid of Windows and you get rid of Excel • ipynb are always shared and versioned; Prefer simple cloud sharing to VCS • Streamline data access functions • Cache long-running code and queries • Develop a common library
39. 39 In Summary...
40. 40 Summary • Agile approach works well in Data Science • Find the lightest suitable tool for a task • Reproducibility is not negotiable • Focus on closing the loop(s)
41. 41 Questions? We are Hiring ! volodymyrk volodymyr.kazantsev@productmadness.com