Content Addressable Storages for Fun and Profit

A lighting talk session from HBaseCon 2012 – May 22, 2012 | San Francisco, CA

This session is a case study of how we used our existing HBase cluster as a content addressable store for BLOBs. We will discuss how we wrote a CAS implementation using HBase as the backend, Scala and Finagle as the application and using caching reverse proxies (i.e. Varnish in our case) for serving BLOBs at scale. The talk will dicuss why content addressable storage is the right pattern for many web use cases, how to foster an existing HBase cluster for better usage of possibly underutilized resources, and operational gotchas to store and serve BLOBs from HBase at scale.

Transcript

1. Content
   Addressable
   Storages
   for fun and profit.
2. Berk D. Demir! @bd
3. I break and fix things at @StumbleUpon.!
4. Problem.
5. Serve lots of static assets
 with low latency
 and high availability.
6. Understanding data.
7. A lot.! Small.! Frequent. 100 million.! 19 kilobytes.! Updates.
8. BLOBs don’t change. They get replaced.! We want to keep all without duplicating.
9. Content
   Addressable
   Store
10. Store immutable content and authoritatively address it with a cryptographic hash.
11. We had ideas.
12. Very bad ideas.
13. Very bad.! ! Shared Storage, i.e., NFS.
14. Bad ideas.
15. Bad.! ! AWS S3, RS Cloud Files, ...! Distributed: AFS, Gluster! HDFS (Oh my!)!
16. Bad ideas.! Take 2
17. o_O! ! Write a distributed, fault tolerant, replicating, multi datacenter, fast, CAS for BLOBs.
18. Reimplementing a lot of things is generally not a good sign.
19. Reuse.! Don’t reimplement.
20. HBase! ! Distributed,! Fault tolerant,! Replicating,! Multi datacenter,! Fast.
21. Immutable rows with compact keys, separated into different column families based on their access patterns.
22. m: d: MD5 16 bytes (SHA-1 20 bytes) Metadata 9 bytes BLOB Many bytes One table to rule them all. MAX_FILESIZE => 20G, VERSION => 1, BLOCKCACHE => true, BLOOMFILTER => ROW ! Pre-split into 512 regions at table creation time.
23. Scala,! Finagle,! asynchbase,! Varnish
24. HTTP has a lot to offer.
25. Verbs! ! GET HEAD PUT DELETE! ! GET /KwIEec5utYGrKmzXYLgFzg HTTP/1.1! Host: b9.sustatic.com
26. Headers! ! Cache-Control: max-age=<1 year>! Last-Modified: <cell timestamp>! Content-MD5: <row key: base64>! ! Content-Disposition: attachment; filename=su.xpi
27. HBase and HTTP are the perfect tools to build simple, reliable, fast data services.
28. Get excited and build things!
29. Thanks.
30. Like the design of this slide deck?! ! Direct your positive feedback to Coda Hale (@coda)