What Data Analysts Wish Application Developers Knew

August 2016 version

Transcript

1. What Data Analysts Wish Applica/on
   Developers Knew
   Alison Stanton
   @alison985
   me@alisonstanton.com
   Pronouns: She/Her/Hers
   #DataAnalystsWish
   

   View Slide

2. #DataAnalystsWish
   

   View Slide

3. Foundational Concepts
   #DataAnalystsWish
   

   View Slide

4. Reconcilia/on
   #DataAnalystsWish
   

   View Slide

5. Developers Are
   Upstream of Data
   Folk
   Business
   •  Priori/zes what gets built
   Developers
   •  Build
   Data
   •  Reports to Business on what
   Business wants to know about the
   build and the resul/ng built thing
   

   View Slide

6. Fictional scenarios based on real life events.
   Names and details have been changed, merged,
   and muddled to protect the guilty.
   

   View Slide

7. Run the World, Inc.
   #DataAnalystsWish
   

   View Slide

8. First Request
   Jackie and Beyonce (Sales
   Team) come to Kerry with
   ques/ons from the client
   about why the demo they
   gave a client with the clients
   data doesn’t match what the
   client is seeing in their trial
   instance.
   #DataAnalystsWish
   

   View Slide

9. The Inves/ga/on Reveals
   •  It’s visible.
   •  It’s not reproducible.
   #DataAnalystsWish
   

   View Slide

10. Kerry's Core Problem
    Missing data. Not
    enough data was saved.
    #DataAnalystsWish
    

    View Slide

11. Kerry's Detailed Problems
    ●  Can’t answer people’s
    ques/ons at all.
    ●  Can’t make decisions based on
    facts.
    ●  There’s no baseline or
    historical informa/on available
    for past informa/on forever.
    #DataAnalystsWish
    

    View Slide

12. Dev. Ques/on
    How do I know what to
    save?
    #DataAnalystsWish
    

    View Slide

13. Answer
    •  Save the dis/nc/ons for
    reconcilia/on.
    •  Think of them as data
    migra/ons.
    •  Keep them in one place.
    #DataAnalystsWish
    

    View Slide

14. Second Request
    bell hooks (Support Director)
    and Marian Wright-Edelman
    (Lawyer) are reques/ng all the
    /mes we responded to a
    customer in IL with ques/ons
    about the guarantee on
    feature Y.
    #DataAnalystsWish
    

    View Slide

15. Can't get to the data.
    #DataAnalystsWish
    Kerry's Core Problem
    

    View Slide

16. Kerry's Detailed Problems
    ●  Ge_ng access
    permissions
    ●  ETL headaches
    #DataAnalystsWish
    

    View Slide

17. Dev. Ques/on
    Where do I save it?
    #DataAnalystsWish
    

    View Slide

18. Answer
    •  A place that
    everyone can get to.
    •  Data is in one place.
    Documenta/on is in
    one place.
    •  One. Singular. One.
    #DataAnalystsWish
    

    View Slide

19. Third Request
    Audre Lorde (CEO) doesn’t get
    her Monday morning
    dashboard. She got an error
    message instead.
    #DataAnalystsWish
    

    View Slide

20. The Inves/ga/on Reveals
    •  Kerry's code hasn't
    changed but the results
    did.
    #DataAnalystsWish
    

    View Slide

21. Kerry's Core Problem
    Produc/on iterated and
    dev team didn’t tell
    anybody.
    #DataAnalystsWish
    

    View Slide

22. Kerry's Detail Problems
    ●  Automated reports
    failed because
    schema changed.
    ●  Meanings changed
    without warning.
    #DataAnalystsWish
    

    View Slide

23. Fourth Issue
    Oprah Winfrey (CMO) starts
    making ad buys because her
    sales report says we have 50
    new customers this month. In
    actuality we have already
    surpassed the goal of 500.
    

    View Slide

24. Dev. Ques/on
    What are the relevant
    pieces of informa/on
    about a deploy to
    communicate to the
    data team?
    #DataAnalystsWish
    

    View Slide

25. Answer
    •  Something is
    changing
    •  Why is it changing?
    •  What is it now and
    what will it change
    to be when
    released?
    •  When is it changing?
    •  ASK ME
    #DataAnalystsWish
    

    View Slide

26. Fiah Request
    Rosa Parks (Finance Director) is preparing
    for an investor audit. She’s reques/ng help
    calcula/ng FIFO on all products.
    Run the
    World, Inc.
    Run
    the
    World,
    Inc.
    #DataAnalystsWish
    

    View Slide

27. The Inves/ga/on Reveals
    •  Aren’t saving history or
    edi_ng diffs.
    #DataAnalystsWish
    

    View Slide

28. Kerry's Core Problem
    Not enough data to
    fulfill the request.
    #DataAnalystsWish
    

    View Slide

29. Kerry's Detailed Problem
    ●  Impossible task
    #DataAnalystsWish
    

    View Slide

30. Dev. Ques/on
    What can we do to
    prevent this in the
    future?
    #DataAnalystsWish
    

    View Slide

31. Answer
    •  created by, created
    at, updated by,
    updated at, status
    •  Separate audit table
    •  What's the risk?
    #DataAnalystsWish
    

    View Slide

32. Sixth Request
    Oprah (CMO) and Angelica
    (CSO) are figh/ng with Harriet
    (COO) about who brings in the
    most business. Every order
    needs to be afributed to
    either the sales or marke/ng
    department.
    #DataAnalystsWish
    

    View Slide

33. Kerry's Core Problem
    Can’t integrate data
    across systems.
    #DataAnalystsWish
    

    View Slide

34. Kerry's Detail Problems
    •  JOINing problems
    because of a lack of
    foreign system keys.
    #DataAnalystsWish
    

    View Slide

35. Dev. Ques/on
    How do I prepare
    without over-
    engineering?
    #DataAnalystsWish
    

    View Slide

36. Answer
    •  Store other systems
    keys whenever
    possible.
    •  Understand,
    maintain, and
    enforce uniqueness.
    •  Tools around
    mapping values.
    #DataAnalystsWish
    

    View Slide

37. Seventh Request
    Toni Morrison (Accounts
    Director) comes over
    complaining that invoices to
    clients are being generated
    incorrectly. Instead of
    charging your biggest
    customer $500,000 the
    invoice was for $5,000.
    #DataAnalystsWish
    

    View Slide

38. Kerry's Core Problem
    There’s a code bug that
    is wri/ng inaccurate
    data.
    #DataAnalystsWish
    

    View Slide

39. Kerry's Detailed Problems
    ●  Lots of /me spent
    playing detec/ve.
    ●  Time spent
    communica/ng with
    developers.
    ●  Time spent fixing.
    ●  Time spent
    communica/ng with
    stakeholders.
    #DataAnalystsWish
    

    View Slide

40. Dev. Ques/on
    How do I minimize
    organiza/onal pain?
    #DataAnalystsWish
    

    View Slide

41. Answer
    •  Fix: Stop the Bleeding.
    •  Communicate: Time
    started, /me ended, what
    was effected and how.
    •  Prevent:
    –  Version control with
    release tags
    –  documenta/on of
    soaware (including the
    bug and how it was
    fixed)
    –  great tests
    –  Code review
    #DataAnalystsWish
    

    View Slide

42. Dear Applica/on Developers,
    Please remember:
    1) Save as much as possible. Anything you will need to know to re-create a
    result is relevant.
    2) We adore standardized, accessible storage that is properly documented.
    3) Please communicate about changes you're making.
    4) Save an audit trail when needed and/or in the context of money.
    5) Assume everything has to be able to talk to each other eventually.
    6) When bugs arise: Fix. Communicate. Prevent.
    Sincerely,
    The Data Folk
    #DataAnalystsWish
    

    View Slide

43. WARNING
    You will be cri2cized for working to
    solve the root of the issue.
    You will probably be cri2cized for talking to
    people instead of only wri2ng code.
    #DataAnalystsWish
    

    View Slide

44. #DataAnalystsWish
    

    View Slide

45. #DataAnalystsWish
    

    View Slide

46. Thank You
    @alison985
    me@alisonstanton.com
    What Data Analysts Wish Applica/on
    Developers Knew
    #DataAnalystsWish
    

    View Slide