Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What Data Analysts Wish Application Developers Knew

Alison
August 27, 2016

What Data Analysts Wish Application Developers Knew

August 2016 version

Alison

August 27, 2016
Tweet

More Decks by Alison

Other Decks in Programming

Transcript

  1. What Data Analysts Wish Applica/on
    Developers Knew
    Alison Stanton
    @alison985
    [email protected]
    Pronouns: She/Her/Hers
    #DataAnalystsWish

    View full-size slide

  2. #DataAnalystsWish

    View full-size slide

  3. Foundational Concepts
    #DataAnalystsWish

    View full-size slide

  4. Reconcilia/on
    #DataAnalystsWish

    View full-size slide

  5. Developers Are
    Upstream of Data
    Folk
    Business
    •  Priori/zes what gets built
    Developers
    •  Build
    Data
    •  Reports to Business on what
    Business wants to know about the
    build and the resul/ng built thing

    View full-size slide

  6. Fictional scenarios based on real life events.
    Names and details have been changed, merged,
    and muddled to protect the guilty.

    View full-size slide

  7. Run the World, Inc.
    #DataAnalystsWish

    View full-size slide

  8. First Request
    Jackie and Beyonce (Sales
    Team) come to Kerry with
    ques/ons from the client
    about why the demo they
    gave a client with the clients
    data doesn’t match what the
    client is seeing in their trial
    instance.
    #DataAnalystsWish

    View full-size slide

  9. The Inves/ga/on Reveals
    •  It’s visible.
    •  It’s not reproducible.
    #DataAnalystsWish

    View full-size slide

  10. Kerry's Core Problem
    Missing data. Not
    enough data was saved.
    #DataAnalystsWish

    View full-size slide

  11. Kerry's Detailed Problems
    ●  Can’t answer people’s
    ques/ons at all.
    ●  Can’t make decisions based on
    facts.
    ●  There’s no baseline or
    historical informa/on available
    for past informa/on forever.
    #DataAnalystsWish

    View full-size slide

  12. Dev. Ques/on
    How do I know what to
    save?
    #DataAnalystsWish

    View full-size slide

  13. Answer
    •  Save the dis/nc/ons for
    reconcilia/on.
    •  Think of them as data
    migra/ons.
    •  Keep them in one place.
    #DataAnalystsWish

    View full-size slide

  14. Second Request
    bell hooks (Support Director)
    and Marian Wright-Edelman
    (Lawyer) are reques/ng all the
    /mes we responded to a
    customer in IL with ques/ons
    about the guarantee on
    feature Y.
    #DataAnalystsWish

    View full-size slide

  15. Can't get to the data.
    #DataAnalystsWish
    Kerry's Core Problem

    View full-size slide

  16. Kerry's Detailed Problems
    ●  Ge_ng access
    permissions
    ●  ETL headaches
    #DataAnalystsWish

    View full-size slide

  17. Dev. Ques/on
    Where do I save it?
    #DataAnalystsWish

    View full-size slide

  18. Answer
    •  A place that
    everyone can get to.
    •  Data is in one place.
    Documenta/on is in
    one place.
    •  One. Singular. One.
    #DataAnalystsWish

    View full-size slide

  19. Third Request
    Audre Lorde (CEO) doesn’t get
    her Monday morning
    dashboard. She got an error
    message instead.
    #DataAnalystsWish

    View full-size slide

  20. The Inves/ga/on Reveals
    •  Kerry's code hasn't
    changed but the results
    did.
    #DataAnalystsWish

    View full-size slide

  21. Kerry's Core Problem
    Produc/on iterated and
    dev team didn’t tell
    anybody.
    #DataAnalystsWish

    View full-size slide

  22. Kerry's Detail Problems
    ●  Automated reports
    failed because
    schema changed.
    ●  Meanings changed
    without warning.
    #DataAnalystsWish

    View full-size slide

  23. Fourth Issue
    Oprah Winfrey (CMO) starts
    making ad buys because her
    sales report says we have 50
    new customers this month. In
    actuality we have already
    surpassed the goal of 500.

    View full-size slide

  24. Dev. Ques/on
    What are the relevant
    pieces of informa/on
    about a deploy to
    communicate to the
    data team?
    #DataAnalystsWish

    View full-size slide

  25. Answer
    •  Something is
    changing
    •  Why is it changing?
    •  What is it now and
    what will it change
    to be when
    released?
    •  When is it changing?
    •  ASK ME
    #DataAnalystsWish

    View full-size slide

  26. Fiah Request
    Rosa Parks (Finance Director) is preparing
    for an investor audit. She’s reques/ng help
    calcula/ng FIFO on all products.
    Run the
    World, Inc.
    Run
    the
    World,
    Inc.
    #DataAnalystsWish

    View full-size slide

  27. The Inves/ga/on Reveals
    •  Aren’t saving history or
    edi_ng diffs.
    #DataAnalystsWish

    View full-size slide

  28. Kerry's Core Problem
    Not enough data to
    fulfill the request.
    #DataAnalystsWish

    View full-size slide

  29. Kerry's Detailed Problem
    ●  Impossible task
    #DataAnalystsWish

    View full-size slide

  30. Dev. Ques/on
    What can we do to
    prevent this in the
    future?
    #DataAnalystsWish

    View full-size slide

  31. Answer
    •  created by, created
    at, updated by,
    updated at, status
    •  Separate audit table
    •  What's the risk?
    #DataAnalystsWish

    View full-size slide

  32. Sixth Request
    Oprah (CMO) and Angelica
    (CSO) are figh/ng with Harriet
    (COO) about who brings in the
    most business. Every order
    needs to be afributed to
    either the sales or marke/ng
    department.
    #DataAnalystsWish

    View full-size slide

  33. Kerry's Core Problem
    Can’t integrate data
    across systems.
    #DataAnalystsWish

    View full-size slide

  34. Kerry's Detail Problems
    •  JOINing problems
    because of a lack of
    foreign system keys.
    #DataAnalystsWish

    View full-size slide

  35. Dev. Ques/on
    How do I prepare
    without over-
    engineering?
    #DataAnalystsWish

    View full-size slide

  36. Answer
    •  Store other systems
    keys whenever
    possible.
    •  Understand,
    maintain, and
    enforce uniqueness.
    •  Tools around
    mapping values.
    #DataAnalystsWish

    View full-size slide

  37. Seventh Request
    Toni Morrison (Accounts
    Director) comes over
    complaining that invoices to
    clients are being generated
    incorrectly. Instead of
    charging your biggest
    customer $500,000 the
    invoice was for $5,000.
    #DataAnalystsWish

    View full-size slide

  38. Kerry's Core Problem
    There’s a code bug that
    is wri/ng inaccurate
    data.
    #DataAnalystsWish

    View full-size slide

  39. Kerry's Detailed Problems
    ●  Lots of /me spent
    playing detec/ve.
    ●  Time spent
    communica/ng with
    developers.
    ●  Time spent fixing.
    ●  Time spent
    communica/ng with
    stakeholders.
    #DataAnalystsWish

    View full-size slide

  40. Dev. Ques/on
    How do I minimize
    organiza/onal pain?
    #DataAnalystsWish

    View full-size slide

  41. Answer
    •  Fix: Stop the Bleeding.
    •  Communicate: Time
    started, /me ended, what
    was effected and how.
    •  Prevent:
    –  Version control with
    release tags
    –  documenta/on of
    soaware (including the
    bug and how it was
    fixed)
    –  great tests
    –  Code review
    #DataAnalystsWish

    View full-size slide

  42. Dear Applica/on Developers,
    Please remember:
    1) Save as much as possible. Anything you will need to know to re-create a
    result is relevant.
    2) We adore standardized, accessible storage that is properly documented.
    3) Please communicate about changes you're making.
    4) Save an audit trail when needed and/or in the context of money.
    5) Assume everything has to be able to talk to each other eventually.
    6) When bugs arise: Fix. Communicate. Prevent.
    Sincerely,
    The Data Folk
    #DataAnalystsWish

    View full-size slide

  43. WARNING
    You will be cri2cized for working to
    solve the root of the issue.
    You will probably be cri2cized for talking to
    people instead of only wri2ng code.
    #DataAnalystsWish

    View full-size slide

  44. #DataAnalystsWish

    View full-size slide

  45. #DataAnalystsWish

    View full-size slide

  46. Thank You
    @alison985
    [email protected]
    What Data Analysts Wish Applica/on
    Developers Knew
    #DataAnalystsWish

    View full-size slide