Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The New Enterprise: RabbitMq, Akka and AWS at Getty Images

The New Enterprise: RabbitMq, Akka and AWS at Getty Images

Slide deck from a NYC Tech Talk Meetup in NYC on how Getty Images uses RabbitMq, Scala/Akka and AWS automation. The images are all available on Getty Images (http://www.gettyimages.com/Account/MediaBin/LightboxDetail.aspx?Id=13728186&MediaBinUserId=4670194)

Michael Hamrah

April 21, 2014

More Decks by Michael Hamrah

Other Decks in Programming


  1. GETTY IMAGES We sell photos (and videos, music, illustrations) Photo:

    Pascal Le Segretain / Getty Images Entertainment
  2. You maybe see a dozen of our images a day?

    Photo: Christopher Polk / Getty Images Entertainment
  3. ENTERPRISE WHAT? More complicated then they need to be Usually

    a mess Many different users A ton of use cases A common problem? Photo: Justin Lewis/Stone
  4. WHAT DO WE WANT? Loosely-coupled systems Easy evolution Leverage OSS

    Simplify Databases Easy infrastructure / HA Horizontal Scale Photo: Michael Hamrah/Moment Open
  5. RABBITMQ Messaging that just works Various messaging semantics We do

    Pub/Sub Publish to Exchange Consumers declare queues, bind to routing keys Ack/Nack messages Photo: 4FR / Vetta
  6. MESSAGES Routing keys: *scope*.*entity*.*action* MessageId: Unique per message CorrelationId: Connect

    messages Header: Primary Key Header: Sender System Identifier Payload: Send Complete Entity (in Bytes) Photo: Michael Steele / Getty Images Sport
  7. HOW TO HANDLE? Get -> Validate -> Process -> Persist

    -> Publish -> Ack Idempotent Messaging == Your BFF Treat as forward-only transactions Photo: Oli Scarff / Getty Images Entertainment
  8. PERSISTENCE: EVENT SOURCING Simplify Persistence Save events, not state Save

    events, not commands Slick: collection-like API Schema Changes Actor == Aggregate Cluster for Uniqueness Photo: Donald Miralle / Getty Images Sport
  9. BUT WHAT ABOUT... Server Configs? Network topology? Databases, Queue Servers?

    Load Balancers? Fine-grain control? Photo: Tyler Olson / Hemera / 360
  10. ALWAYS BE LAUNCHING Define your topology Pre-bake a server Launch

    DIY PaaS Photo: Cameron Spencer / Getty Images Sport
  11. PACKER SUPERCHARGED VAGRANT Builds Amis, Docker VMware, VBox, more. Supports

    Chef/Puppet/Scripts Minimize startup times Predictable Photo: Scott Heavey / Getty Images Sport
  12. CLOUDFORMATION INFRASTRUCTURE AS CODE (JSON) Defines network, elb's, asg's, r53's,

    db's, everything Pass data via cloud-init Environment variables are helpful Launch multiple stacks, destroy at will Photo: Steve Allen/Digital Vision
  13. JENKINS DOES ALL Merge = AMI Packer via Puppet Passed

    to CFN template Update/Create Stack Roll in new AMIs Cutover DNS Enjoy Photo: Chris Hyde / Getty Images Sport