Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Data Antipatterns - All Your Base

C64a0152c9b0928e62d88f0bb5eb8138?s=47 Ines Sombra
October 18, 2013

Data Antipatterns - All Your Base

Are you running a database in the cloud? Worried that you're doing it wrong?

Engine Yard supports a broad set of databases with flexibility for customers to modify and configure. However, freedom to adapt and extend standard functionality comes with unexpected negative consequences: modifications can seriously affect durability and performance. I've observed common problems, patterns and best practices with big (and not so big) data. I'll highlight the most common pitfalls and discuss how to avoid them.

Video for this talk is available here: http://vimeo.com/83755776


Ines Sombra

October 18, 2013

More Decks by Ines Sombra

Other Decks in Technology



  2. ines @ Engine Yard.com @Randommood And I’m a happy dog!

    I N E S S O M B R A I work with Databases
  3. Engine Yard

  4. ZOMG, the horror!

  5. .BACKUPS yes, we are going there

  6. “I know you. You know you. And I know you

    know that I know you” White Goodman (no relationship to White October)
  7. Boring Definition #1 Backups Copy and archiving of data Goal

    is to restore the state of a DB Many types - blah
  8. Anti-Pattern #1 Taking too many backups Not free, they requires

    resources Full backup every hour, really? What about backup retention?
  9. Anti-Pattern #2 Taking too few backups Enough to minimize the

    risk of data loss due to corrupted backup files yes,
  10. The untested backup Anti-Pattern #3

  11. Doing backups right Logically test backups Errorless restore is not

    enough. Test logical data too
  12. Doing backups right Know your types & tools Take logical

    and binary backups Continuous archiving & hot backup utilities
  13. Doing backups right Practice restores Backups alone do not constitute

    DR. Have a plan & practice it Server extensions and configuration matter when restoring
  14. “I want a ridiculously good looking Database” Derek Zoolander (honestly,

    Ben Stiller rules)
  15. Obvious statement #1 Many DB choices

  16. Cargo culting your database Anti-Pattern #4 Failure to understand use

    case, strengths & weaknesses of a new database
  17. RDBMS for Session Data Anti-Pattern #5 Often means at least

    one write per request Any DB issue/task may cause app to hang Tables have a tendency to bloat
  18. Modeling, it’s all the same Anti-Pattern #6

  19. Data Model Consistency needs Availability needs Scaling needs Operational story

    & cost Doing it right Know your needs
  20. Doing it right Spike it, forealsies Spike it with your

    data and traffic. Best way to gain operational experience
  21. Doing it right Leverage new features Relational databases are getting

    quite versatile Evaluate clustered MySQL options
  22. We have a cloud deployment! Happy team on shipping day,

    lmfao if you don’t celebrate like this
  23. Cloud-based databases, they are real Obvious statement #2 Databases can

    live in the cloud quite well Many IaaS, PaaS, & DBaaS options Easy to get started & may be economical
  24. Where did my instance go? Anti-Pattern #7

  25. Anti-Pattern #8 Cloud, it’s just like hardware It’s not. Cloud

    resources are virtualized Capacity planning and monitoring matter. A lot
  26. Anti-Pattern #9 Shit doesn’t happen You are not immune to

    infrastructure failures. Plan for it
  27. Anti-Pattern #10 Storage is the same Instance storage is not

    persisted (use EBS) Data locality matters Don’t run your cloud DBs too hot!
  28. Doing cloud right Know your cloud deployments Replication in the

    cloud is a must-have Put DB master & replicas in different AZs
  29. Doing cloud right Learn high availability & disaster recovery Get

    good at replica promotions (some work involved) Understand and invest in DR/HA. Know your options
  30. Doing cloud right Know your system Invest in monitoring Know

    your data distribution & querying patterns Know baseline behavior
  31. Questions?