Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Observability: Superpowers for Developers

Observability: Superpowers for Developers

Who wants to spend time dreaming about the ability to leap tall buildings with a single bound, when we can recast stories we live day to day as powers of our own... and improve our own lives in a practical way? When observability is folded into the development process itself, it represents the potential for a beautifully virtuous cycle: production stops being just where our development code runs into issues, and it becomes where part of our development process *lives*.

A58e27f1ad31899d3b4385b0d7689913?s=128

Christine Yen

June 04, 2019
Tweet

More Decks by Christine Yen

Other Decks in Technology

Transcript

  1. observability: Superpowers for Developers @cyen @honeycombio

  2. EXPOSITION SETTING THE SCENE

  3. DEV Christine

  4. DEV WRITE → TEST → COMMIT → WRITE → TEST

    → COMMIT → WRITE → TEST → COMMIT → WRITE → TEST → COMMIT → WRITE → TEST → COMMIT → WRITE → TEST → COMMIT → WRITE → TEST → COMMIT → WRITE → TEST → COMMIT
  5. DEV OPS WRITE → TEST → COMMIT → RELEASE →

    DEBUG → FIX
  6. "Works on my machine" DEV "The only good diff is

    a red diff" OPS
  7. —Subbu Allamaraju, Expedia, Feb 2019
 https://m.subbu.org/incidents-trends-from-the-trenches-e2f8497d52ed "Observation 1: Change is

    the most common trigger"
  8. APP API GATEWAY USER MGMT BILLING WEB UI PARTNER MGMT

    PAYMENTS INTERNAL WEB UI TXN MGMT NOTIFICATION SYSTEM REST API REST API REST API REST API REST API REST API THEN NOW
  9. "Works on my machine" DEV "The only good diff is

    a red diff" OPS
  10. None
  11. THE FIRST WAVE: THE SECOND WAVE: OPS DEV teaching devs

    to own code in production getting ops folks to code
  12. WHY DEVS? CHARACTER BUILDING

  13. EXPECTED ACTUAL vs

  14. --- FAIL: TestUnitTest (0.00s) talk_test.go:10: -- actual: 5 (type int)

    expected: 4 (type int) ACTUAL EXPECTED
  15. WRITE → TEST → COMMIT → RELEASE → OBSERVE TEST

    OBSERVE DEV
  16. DEV OPS

  17. WRITE → TEST → COMMIT → RELEASE → OBSERVE TEST

    OBSERVE DEV OPS
  18. WHAT DOES "O11Y AS A SUPERPOWER" MEAN? DISCOVERING-NEW-POWERS MONTAGE

  19. ⬜ SPEAKS THE LANGUAGE OF PROD ⬜ SEES THROUGH PROD

    TO CODE ⬜ EXPERIMENTS+CHECKS HYPOTHESES
  20. PROD ME SPEAKS THE LANGUAGE OF PROD

  21. ▸ Tools support the vocabulary natural to devs SPEAKS THE

    LANGUAGE OF PROD CPU utilization AWS availability zone kafka partition Cassandra hostname
  22. SPEAKS THE LANGUAGE OF PROD ▸ Tools support the vocabulary

    natural to devs CPU utilization AWS availability zone kafka partition Cassandra hostname $YOUR_BIZ-relevant ID build ID API endpoint HTTP verb
  23. ▸ Tools support the vocabulary natural to devs ▸ Tools

    support (custom) high-cardinality data SPEAKS THE LANGUAGE OF PROD payload size client OS build ID API endpoint HTTP verb time to render $YOUR_BIZ-relevant ID
  24. ▸ Tools support the vocabulary natural to devs ▸ Tools

    support (custom) high-cardinality data SPEAKS THE LANGUAGE OF PROD
  25. ▸ Tools support the vocabulary natural to devs ▸ Tools

    support (custom) high-cardinality data ▸ Instrumentation under constant evolution and tinkering SPEAKS THE LANGUAGE OF PROD dev prod
  26. SEES THROUGH PROD TO CODE PROD ME CODE

  27. SEES THROUGH PROD TO CODE ▸ Tracing: not just for

    concurrent or distributed systems
  28. SEES THROUGH PROD TO CODE ▸ Tracing: not just for

    concurrent or distributed systems ▸ Describes describe shapes/workflows that feel familiar
  29. SEES THROUGH PROD TO CODE

  30. SEES THROUGH PROD TO CODE ▸ Tracing: not just for

    concurrent or distributed systems ▸ Describes describe shapes/workflows that feel familiar ▸ Support natural investigative flow
  31. EXPERIMENTS + CHECKS HYPOTHESES PROD ME CODE

  32. EXPERIMENTS + CHECKS HYPOTHESES ▸ Understand what "normal" is for

    the system
  33. EXPERIMENTS + CHECKS HYPOTHESES ▸ Understand what "normal" is for

    the system ▸ Production as a testbed for in-development code
  34. EXPERIMENTS + CHECKS HYPOTHESES ▸ Understand what "normal" is for

    the system ▸ Production as a testbed for in-development code ▸ Feature flags + observability = WRITE → TEST → RELEASE → OBSERVE OBSERVE OBSERVE OBSERVE OBSERVE OBSERVE
  35. ☑ SPEAKS THE LANGUAGE OF PROD ☑ SEES THROUGH PROD

    TO CODE ☑ EXPERIMENTS+CHECKS HYPOTHESES
  36. BUILDING A SHARED BRAIN BUILDING A SUPERHERO CREW

  37. PAST PRESENT FUTURE TEAM BUILDING A SHARED BRAIN

  38. "Leaps of intuition akin to magic"? EMPOWERING NON-EXPERTS active vs

    passive knowledge transfer Learnable + shareable
  39. … ANTICLIMAX AKA, WHERE’S THE BATTLE SCENE?

  40. REDUCE THE BATTLES WE HAVE TO FIGHT

  41. (OR ELSE RISK BURNOUT)

  42. FINAL VOICEOVER SUPERHERO RETROSPECTIVE

  43. OPS: DEVS: embrace observability, bring production closer to development. share

    the great responsibility
 (and great power!) YOU CAN HAVE SUPERPOWERS NOW.
  44. WRITE → TEST → COMMIT → RELEASE → OBSERVE TEST

    OBSERVE DEV OPS
  45. ASK NEW QUESTIONS SHIP BETTER SOFTWARE thanks! q’s? @cyen @honeycombio

    ON TO THE NEXT ADVENTURE…