Danse avec les unicorns : la data science en agile, de l'exploration à l'adoption

5dd8b91d96af5a1e33962df41c1a4d20?s=47 Esprit Agile
December 06, 2018

Danse avec les unicorns : la data science en agile, de l'exploration à l'adoption

Conférence par Wassel Alazhar lors de l'Agile Tour Aix-Marseille 2018.

"La data science sort enfin des grottes des scientifiques. Tout le monde est convaincu que cette discipline changera la donne.
Les outils et les plateformes foisonnent et les startups qui ont fait ce pari sont devenues des unicorns.

Cependant, construire des applications qui apportent une réelle valeur ajoutée reste un vrai challenge. Plusieurs projets prometteurs n'ont pas réussi à dépasser le stade du PoC.

L'année dernière, j'ai accompagné mon client dans son aventure avec une "licorne" californienne qui produit une plateforme de Big data analytics et AI.
Dans cette session, nous allons voir comment faire une data science agile pour l'ajout constant de valeur.

On s'intéressera à :

– l'exploration de la valeur métier
– la collaboration entre différents domaines d'expertise (data science, data integration, app development, UX…)
– la qualité du produit dans un domaine exploratoire

Une agilité à tous les étages !"

5dd8b91d96af5a1e33962df41c1a4d20?s=128

Esprit Agile

December 06, 2018
Tweet

Transcript

  1. Dances with unicorns Agile datascience from exploration to adoption

  2. Un grand merci à nos sponsors et partenaires

  3. Oman 11.30 AM

  4. Paris 9.45 AM

  5. <24h From user’s feedback to production

  6. One month later...

  7. This project has been abandoned

  8. That’s me! Wassel Alazhar Consultant, developer, problem solver @wasselovski https://github.com/jcraftsman

  9. Agenda • The full story • What went wrong? •

    What did we learn? ◦ How to bring value from datascience ◦ Explore and build ◦ Efficient collaboration ◦ Product quality • Why this talk? • Takeaways
  10. The full story

  11. A global energy leader

  12. A global energy leader Produce Deliver SELL

  13. A global energy leader Produce Deliver SELL Sensors everywhere! All

    along the value chain
  14. The problem to solve Produce Deliver SELL Sensors everywhere! All

    along the value chain
  15. The problem to solve Two new generation power plants They

    are exactly the same but... Twin A Twin B
  16. The problem to solve Twin B is way more performant

    (i.e., makes money) Twin A Twin B
  17. The solution Datascience can help identifying better operational models for

    the power plants
  18. The solution BIG DATA + DATA SCIENCE = MAGIC

  19. The partner

  20. The partner A unicorn is a privately held startup company

    valued at over $1 billion
  21. The bill But wait… How much is that? Nevermind. It’s

    all on me! It’s called innovation. Great!
  22. The team Data engineers Data scientists App developers

  23. To the Silicon valley Data engineers Data scientists App developers

  24. Week after week… Demo after demo

  25. It couldn’t be any better

  26. SURPRISE Now, it’s all yours! All you have to pay

    for is the run. Oh! No, thanks. I’m out of it.
  27. Deception

  28. What went wrong?

  29. Building a software!

  30. What was the Problem to solve? Do you remember the

    twin power plants? Twin A Twin B
  31. Not what we’ve expected... Problem solved explained quickly No actionable

    findings
  32. Instead we have delivered features! Degradation analysis Anomaly detection The

    software can detect dust in the steam turbine! PCA???
  33. Feature ≠ VALUE

  34. What did we learn?

  35. Happy ending stories... Predictive maintenance Smart buildings Ice detection Heating

    and cooling efficiency
  36. Business use case discovery Don’t start with a software! Explore

    Observe Confirm hypothesis or not! Discover
  37. Business use case discovery Don’t explore in a dark lab!

    Get feedback!
  38. Business use case discovery A python notebook is not a

    software! It’s a tool for a study!
  39. From study to product delivery Business use case located? Build!

    VALUE
  40. Product delivery Not like this!

  41. Wait, what does datascience look like in 2018? How would

    you write a program for puppy recognition?
  42. Wait, what does datascience look like in 2018? You can:

    • Try to define what a puppy face is • Code all these rules! Or, use Machine learning: • Show a lot of puppy faces examples! You don’t need to tell the algorithm what to do. All you need is to show it a lot of examples!
  43. Wait, what does datascience look like in 2018? Take care

    of your examples (data pipeline) Verify the results (predictions)
  44. Putting it all together Discovery: Given a real world pictures

    sample, would it be possible to recognize a puppy face? The answer is 86% yes, 13% muffins, 1% unknown. Product: Play a dog kibble comercial whenever a puppy picture is displayed!
  45. Explore and build Explore: • Gathering data • Cleaning data

    • Feature engineering • Defining model • Training • Predicting the output => Discover what you are able to do with your data Build: • Data acquisition • Data filtering • Use model configuration • Use model • Training (or use a train set) • Predicting the output => Steadily bring value from your data
  46. Explore and build iteratively Explore: • Gathering data • Cleaning

    data • Feature engineering • Defining model • Training • Predicting the output => Discover what you are able to do with your data Build: • Data acquisition • Data filtering • Use model configuration • Use model • Training (or use a train set) • Predicting the output => Steadily bring value from your data
  47. Explore and build iteratively Explore Build Business use case discovery

    Product delivery
  48. Product delivery You’re not done with datascience! They should build

    together!
  49. Building together Code review When? All the time! Who? Everyone!

    Why? Quality, collective ownership and joy!
  50. Building together Pair programming When? All the time! Who? Everyone!

    Why? Quality, collective ownership and joy!
  51. Building together Mob programming When? Whenever you start something new

    or complex. Who? Everyone! Why? Collective intelligence, collective ownership, quality and joy!
  52. Building together TDD Let’s be serious! When? Whenever you change

    the product’s behaviour. Who? Everyone working on the product! Why? Collective intelligence, collective ownership, quality and joy!
  53. Building together TDD Have you ever met a data scientist

    who write unit tests and refactor? I did! :) It’s hard to imagine doing TDD during an exploratory work though! (i.e., when the target observable behaviour is not yet defined)
  54. Product delivery Spikes and user stories

  55. Product delivery essentials Don’t lose time repeating boring stuff! Automate!

    Make data available for everyone! Don’t treat your infra like pets! Destroy and rebuild! Don’t over-engineer though!
  56. Product adoption Stay close to the users! Don’t plan too

    many features! Incorporate feedback!
  57. What is agile anyway?

  58. Can datascience be agile? It’s still true! Even for: •

    Big data • AI • Datascience
  59. Why this talk?

  60. Myths about datascience Well… Things have slightly changed since then…

    But not that much!
  61. Myths about datascience

  62. Myths about datascience

  63. Unicorns

  64. New unicorns - Same old stories You should draw your

    entire model before you start coding! Open a ticket! You need to hire a machine learning engineer!
  65. Takeaways!

  66. Takeaways! Make people together! Business value discovery => product delivery

    Explore and build iteratively Agile is still: • Short feedback • Small increments • Take engineering seriously work learn
  67. OCTO © 2018 - Reproduction interdite sans autorisation écrite préalable

    67 OCTO Provence recrute ! C’EST AVANT TOUT UN ÉTAT D’ESPRIT START-UP APPUYÉ PAR DES EXPERTISES TECH, AGILE & CHANGE POUR ACCOMPAGNER DIGITALE TRANSFORMATION NOS CLIENTS DANS LEUR Contactez-nous sur PROVENCE@OCTO.COM