Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Managing Terrabytes on AWS

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Baruch Sadogursky Baruch Sadogursky
February 19, 2013
250
0

Managing Terrabytes on AWS

Talk by Fred Simon at DevNexus 2013

Avatar for Baruch Sadogursky

Baruch Sadogursky

February 19, 2013

More Decks by Baruch Sadogursky

See All by Baruch Sadogursky
Silicon Valley Chainsaw Massacre (how I spent my last Friday night on-call) as presented at DevOops 2017
Avatar for Baruch Sadogursky jbaruch
0
800
DevOps @ scale — greek tragedy in three acts as presented at Devoops 2017
Avatar for Baruch Sadogursky jbaruch
0
630
Building a Massively Scalable Cloud Service from the Grounds Up
Avatar for Baruch Sadogursky jbaruch
0
340
Groovy DSL для plugin-ов - Расширения человеческим языком
Avatar for Baruch Sadogursky jbaruch
0
420
Найти Сятой Грааль, и оттестировать его как следует
Avatar for Baruch Sadogursky jbaruch
1
450
Plugging the users in - extend your application with pluggable Groovy DSL
Avatar for Baruch Sadogursky jbaruch
0
280
Open/Closed Software - Developing freemium application using Spring Framework
Avatar for Baruch Sadogursky jbaruch
0
280
Search for the Holy Grail (and test it once found)
Avatar for Baruch Sadogursky jbaruch
0
180
Разработка модулярного софта — добро пожаловать в АД!
Avatar for Baruch Sadogursky jbaruch
0
150

Featured

See All Featured
The innovator’s Mindset - 
Leading Through an Era of Exponential Change - McGill University 2025
Avatar for Jean-Marc De Jonghe jdejongh
PRO
1
370
Reality Check: Gamification 10 Years Later
Avatar for Sebastian Deterding codingconduct
0
2.3k
Mozcon NYC 2025: Stop Losing SEO Traffic
Avatar for Sam Torres samtorres
1
570
Building a A Zero-Code AI SEO Workflow
Avatar for Ian Lurie portentint
PRO
0
760
The Mindset for Success: Future Career Progression
Avatar for Greg Gifford greggifford
PRO
0
530
How People are Using Generative and Agentic AI to Supercharge Their Products, Projects, Services and Value Streams Today
Avatar for Helen Beal helenjbeal
1
350
The Power of CSS Pseudo Elements
Avatar for Geoffrey Crofte geoffreycrofte
82
6.6k
Keith and Marios Guide to Fast Websites
Avatar for Keith Pitt keithpitt
413
23k
Conquering PDFs: document understanding beyond plain text
Avatar for Ines Montani inesmontani
PRO
4
3.2k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
Avatar for Michelle Schulp Hunt marktimemedia
31
2.9k
From Legacy to Launchpad: Building Startup-Ready Communities
Avatar for Dug Song dugsong
0
350
Navigating the Design Leadership Dip - Product Design Week Design Leaders+ Conference 2024
Avatar for Andy Polaine apolaine
2
460

Transcript

  1. None
  2. None
  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. None
  10. None
  11. None
  12. None

  13. Request Storage File Listing Checksum Storage

  14. Request Storage File Listing Checksum Storage Metadata requests Cache check,

    FS request DB request

  15. Request Storage File Listing Checksum Storage Metadata requests Cache check,

    FS request DB request HEAD requests Cache check, FS request DB request

  16. Request Storage File Listing Checksum Storage Metadata requests Cache check,

    FS request DB request HEAD requests Cache check, FS request DB request GET request Cache check, FS request DB + FS request

  17. Request Storage File Listing Checksum Storage Metadata requests Cache check,

    FS request DB request HEAD requests Cache check, FS request DB request GET request Cache check, FS request DB + FS request PUT request FS request DB + FS request
  18. None
  19. None
  20. None
  21. None
  22. None
  23. None
  24. None
  25. None
  26. None
  27. None
  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. None
  35. None
  36. None
  37. None

  38. EBS + LVM = WIN!

  39. /data 250GB 250GB

  40. /data 250GB 250GB 250GB 250GB

  41. /data 250GB 250GB 250GB 250GB 250GB 250GB

  42. None
  43. None
  44. None
  45. None
  46. None
  47. None
  48. None
  49. None
  50. None

  51. Backup files

  52. Backup files Backup database

  53. Backup files Backup database Keep them in sync!

  54. Backup files Backup database Keep them in sync!

  55. Backup only deltas

  56. Backup only deltas Manage threads to ec2 efficiently

  57. Backup only deltas Manage threads to ec2 efficiently Keep the

    data isolated

  58. Backup only deltas Manage threads to ec2 efficiently Keep the

    data isolated
  59. None
  60. None
  61. None
  62. None
  63. None
  64. None

  65. Customer EBS Customer EBS Customer EBS

  66. De- duped EBS HA Storage Customer EBS Customer EBS Customer

    EBS
  67. None

  68. De- duped Glacier De- duped EBS Backup HA Storage Customer

    EBS Customer EBS Customer EBS
  69. None
  70. None
  71. None
  72. None
  73. None

  74. Create empty Artifactory instance

  75. Create empty Artifactory instance Populate from back storage on demand

  76. Create empty Artifactory instance Populate from back storage on demand

    Drink Cosmopolitan

  77. Conclusions time 1. Be lazy.

  78. None

  79. Conclusions time 1. Be lazy. 2. Be smart.

  80. Conclusions time 1. Be lazy. 2. Be smart. 3. Be

    agile.

  81. Drink Cosmopolitan