Maslow's Hierarchy of Database Needs

Maslow's Hierarchy of Database Needs

It’s easy to say, you should always automate everything, monitor everything, and instrument every inch of your data infra. But overengineering in advance of your needs can be just as costly as the reverse, particularly for startups. Engineering cycles are scarce. How do you decide where to spend them?

Enter Maslow’s hierarchy of needs—for databases.

For humans, Maslow’s hierarchy of needs is a pyramid of desires that must be satisfied for us to flourish: survival, safety, love and belonging, esteem, and self-actualization. Each level depends on the preceding ones—we need survival before safety, safety before love and belonging, etc.

Really, databases aren’t so different from you and me. They need:

Survival: Do you even have a data store? How should you decide what storage systems to run? Is it up? Is it alive? Do you have backups? Are they valid?

Safety: Are there multiple live copies? Are they geographically distributed? What is your failover story? Are your humans redundant too?

Love and belonging: Are your databases first-class citizens of your engineering processes, and do they share config management and tooling with the rest of the org? Are schema changes defined in code and revertable? Have you eliminated special snowflakes?

Esteem: Can your observability stack surface problems before they impact production? Can you correlate events across the stack? Can you automatically remediate common failures without human intervention? How do your observability requirements evolve as your org matures?

Self-actualization: Is your data store its best possible self, and what does that even mean for your org? (The self-actualized, maturely instrumented storage layer for a website with 1 billion users will look very different from the self-actualized storage layer for a young and highly volatile startup environment.) How can you assess your appetite for risk, your stage of development, and the layers of process you should invest in organizationally at each stage?

Ac734fc32781678475b577944bb5a9ae?s=128

Charity Majors

June 23, 2016
Tweet

Transcript

  1. Maslow’s Hierarchy of Needs For Databases Charity Majors, CTO honeycomb.io

    @mipsytipsy
  2. Maslow’s Hierarchy of Needs For Databases Charity Majors, CTO honeycomb.io

    @mipsytipsy
  3. Dr. Abraham Maslow, Psychologist real talk about storage systems …

    for non-DBAs
  4. “Database Reliability Engineering” with Laine Campbell

  5. Dr DevOps is in the house breakin’ down yr silos

  6. Maslow’s hierarchy of needs

  7. Databases: They’re Just Like Should Be Just Like Other Services!

  8. “self-actualized database” • Empowering for developers • Resilient to common

    failures • Friendly to operations (debuggable, understandable)
  9. The “right database” is the one that empowers you to

    achieve your mission.
  10. Survival

  11. Selecting a storage system • Choose boring technology, when you

    can • Reuse solutions. Resist software sprawl • Spend innovation tokens only on key differentiators h/t @mcfunley
  12. • If your company is very young, optimize for velocity

    and developer productivity. • The more mature your company is, the more operational impact over the long term trumps all.
  13. Safety

  14. Backups • How much data can you afford to lose?

    • Monitor backup freshness • Archive remotely. Test your restore process.
  15. If you are not regularly testing restores, you have Schrodinger’s

    backups.
  16. Replication • Multiple live copies • Consider write concern, replication

    factor, quorum, votes, dedicated backup nodes … • Distribute across AZs, regions, or DCs
  17. Failover • What happens when any node dies? • Practice

    and document under controlled circumstances • Human SPOFs are just as bad as machine SPOFs
  18. Ponder your path to horizontal scalability Forecasting more than 10x

    growth is mostly impossible Just try to not screw over your future self.
  19. Love & Belonging

  20. “Show me the DevOps!” Making your data a first-class citizen

    of both dev and ops processes
  21. Dev, Ops, DBA • Use the same code review and

    deploy processes • Use the same infrastructure provisioning and config management • Data-land should feel familiar and intuitive, not alienating or “special”
  22. Embrace failure

  23. Guard Rails • Backups, restores. Verified. • Unit tests •

    Boring failovers • Shared on call rotations • Network isolation between prod & other • Don’t get pissy when people mess up, help them fix it • Don’t swoop in and do it all yourself
  24. Right-sizing your automation • Indexing • Scaling up or down

    • Failover • Killing bad queries
  25. Esteem

  26. Your services need: • Observability • Instrumentation • Graceful degradation

  27. Your humans need: • Tools for debugging • Checklists, documentation

    • Tractable levels of graphs and alerting
  28. Lifecycle of instrumentation Is it up? Is it slow? Canned

    graphs for system and db metrics Hand crafted dashboards Lots of outages Collection of “heroic” debugging commands Automated query profiling / analysis Auto-remediation Unique request ids, full-stack tracing Realtime exploratory tools Predictive analysis / precognition skip or change shrink if we’re at 20 min or less, include this slide use this for silvia’s talk?
  29. Golden rules for alerting • Emphasize end-to-end checks on key

    indicators • Health of the service, not individual nodes • Page only if actionable • Track auto-remediation events • Shared dev/ops rotation by service owners
  30. None
  31. Self-Actualization

  32. basically … (credit: @zakgreant)

  33. Self-actualized database • Your data is safe and recoverable •

    Resilient to common errors • Understandable, debuggable • Shares processes and tooling with the rest of the stack • Empowers you to achieve your mission.
  34. in conclusion Don’t be scared. Mind your damn backups. Everything

    else will probably be ok.