The New Enterprise: RabbitMq, Akka and AWS at Getty Images

Slide deck from a NYC Tech Talk Meetup in NYC on how Getty Images uses RabbitMq, Scala/Akka and AWS automation. The images are all available on Getty Images (http://www.gettyimages.com/Account/MediaBin/LightboxDetail.aspx?Id=13728186&MediaBinUserId=4670194)

Transcript

1. THE NEW ENTERPRISE MIGRATING TO AKKA, RABBITMQ AND AWS / Michael Hamrah @mhamrah
2. GETTY IMAGES We sell photos (and videos, music, illustrations) Photo: Pascal Le Segretain / Getty Images Entertainment
3. 200,000 Contibutors Photo: Nisian Hughes/Stone
4. 185 Countries Photo: Jeff J Mitchell / Getty Images News
5. 300 Image Partners Photo: Martin Puddy/Stone
6. Large Archive Photo: Bob Gomel/The LIFE Images Collection
7. You maybe see a dozen of our images a day? Photo: Christopher Polk / Getty Images Entertainment
8. Dev teams in Seattle, NYC, Calgary, Madison Scala, Ruby, Go, C#, Java, PHP
9. ENTERPRISE WHAT? More complicated then they need to be Usually a mess Many different users A ton of use cases A common problem? Photo: Justin Lewis/Stone
10. WHAT DO WE WANT? Loosely-coupled systems Easy evolution Leverage OSS Simplify Databases Easy infrastructure / HA Horizontal Scale Photo: Michael Hamrah/Moment Open
11. BUT HOW? RabbitMq Scala + Akka CQRS/Event Sourcing
12. RABBITMQ Messaging that just works Various messaging semantics We do Pub/Sub Publish to Exchange Consumers declare queues, bind to routing keys Ack/Nack messages Photo: 4FR / Vetta
14. MESSAGES Routing keys: *scope*.*entity*.*action* MessageId: Unique per message CorrelationId: Connect messages Header: Primary Key Header: Sender System Identifier Payload: Send Complete Entity (in Bytes) Photo: Michael Steele / Getty Images Sport
15. ASSET DOMAIN Photo: Ilya Terentyev / Vetta
16. HOW TO HANDLE? Get -> Validate -> Process -> Persist -> Publish -> Ack Idempotent Messaging == Your BFF Treat as forward-only transactions Photo: Oli Scarff / Getty Images Entertainment
17. AKKA: LET IT CRASH Actor Model Lightweight Concurrent Resilient Photo: Vladimir Rys / Bongarts / Getty Images
18. AKKA Runtime for objects Natural fit for sub-systems Photo: Harry How / Getty Images Sport
19. ACTOR HIERARCHY Photo: Paul Taylor / Stone
20. ACTOR DEPENDENCIES ActorRef = Existing Props = Create Great Test Framework Photo: 4FR/Vetta
22. PERSISTENCE: EVENT SOURCING Simplify Persistence Save events, not state Save events, not commands Slick: collection-like API Schema Changes Actor == Aggregate Cluster for Uniqueness Photo: Donald Miralle / Getty Images Sport
23. CLEAN ARCHITECTURE / Robert "Uncle Bob" Martin Vimeo
24. DEPLOY! AUTOMATE! TO THE CLOUD! Photo: Deborah Harrison / Photographer's Choice RF
25. BUT WHAT ABOUT... Server Configs? Network topology? Databases, Queue Servers? Load Balancers? Fine-grain control? Photo: Tyler Olson / Hemera / 360
26. WHAT IF YOU NEVER UPDATED A SERVER? Photo: Ronald Martinez / Getty Images Sport Classic
27. ALWAYS BE LAUNCHING Define your topology Pre-bake a server Launch DIY PaaS Photo: Cameron Spencer / Getty Images Sport
28. PACKER SUPERCHARGED VAGRANT Builds Amis, Docker VMware, VBox, more. Supports Chef/Puppet/Scripts Minimize startup times Predictable Photo: Scott Heavey / Getty Images Sport
29. CLOUDFORMATION INFRASTRUCTURE AS CODE (JSON) Defines network, elb's, asg's, r53's, db's, everything Pass data via cloud-init Environment variables are helpful Launch multiple stacks, destroy at will Photo: Steve Allen/Digital Vision
30. JENKINS DOES ALL Merge = AMI Packer via Puppet Passed to CFN template Update/Create Stack Roll in new AMIs Cutover DNS Enjoy Photo: Chris Hyde / Getty Images Sport
31. THE END Photo: Denis O'Regan / Premium Archive