Building Modular Apps with Ruby & AWS

A high-level look at how AWS can be used to break monolithic apps into distributed components.

Resources:

* Official AWS Ruby gem (GitHub) https://github.com/aws/aws-sdk-ruby
* Documentation: http://docs.amazonwebservices.com/AWSRubySDK/latest/

Transcript

1. Building Modular Apps with Ruby & AWS Kyle Fox / @kylefox YEGRB / November 20, 2012
2. Carbonmade in a nutshell • Artists signup & build a portfolio (back-end) • People visit artists’ portfolios (front-end)
3. Artist Manage Portfolio Visitors View Portfolio Monolith! Massive .NET app & database server Admin Junk Reports, customer support, etc.
4. The Problems I got 99 problems and they’re all architecture related.
5. Massive Rackspace Server • Dedicated = Expensive • Slow to provision new hardware • Resource contention
6. Monolithic Design • Single point of failure • Bottleneck is EVERYTHING • Break once, break everywhere • Tight coupling = house of cards
7. Workflow Issues • Hard to find .NET developers • Editing files over FTP • Deploys required compilation, or something dumb like that • Difficult to write tests
8. what to do?
9. let’s try a SERVICE ORIENTED ARCHITECTURE
10. A fancy way of saying Let’s break the app into a bunch of isolated, single-responsibility components! And have them work together over black-box APIs. Amazon’s Web Services are great for this purpose.
11. Advantages of a SOA • Avoids a single point of failure • Easier to scale parts that need scaling • Makes infrastructure refactoring a breeze • Decoupled by design • Easier to rebuild in phases than all at once
12. AWS we’ll use • Elastic Compute Cloud (EC2) • Server Virtualization & Load Balancing • Simple Storage Service (S3) • Persistent Data Storage • CloudFront (CF) • Content Delivery Network • Simple Queue Service (SQS) • Message Queue • Simple Notification Service (SNS) • Broadcast Notifications (pubsub)
13. Step #1 Remove public portfolio rendering from monolithic .NET app
14. Visitors View Portfolio .NET Database & Back-end tools Sinatra •EC2 •Stateless (no database) •Reads from API •Renders data in Liquid templates Get /projects/123 {project: ... }
15. Good things • Moved some traffic off back-end • Designed an API • Moved templates to Liquid! • Git & automated deployment
16. Bad things • Single point of failure • No caching • No failover / redundancy • App logic (data fetching & theme rendering) is embedded in a web framework. Gross.
17. Step #2 Many Sinatra apps & carbonmade gem
18. EC2 Load Balancer Visitors View Portfolio .NET Database & Back-end tools Sinatra Sinatra Sinatra Cache Huge membase cluster Carbonmade Gem High-memory EC2 instances Multiple Portfolio Servers
19. EC2 Load Balancer Visitors View Portfolio .NET Database & Back-end tools Sinatra Sinatra Sinatra Cache Huge membase cluster Carbonmade Gem Amazon SQS Cache invalidation messages Queue message when portfolio updated Poll for messages Delete from cache Cache Invalidation
21. Step #3 Build a Theme Park ...wait, huh?
22. Designers shouldn’t need to deploy to server clusters When all they’re doing is changing some CSS
23. ...well, they should but they refused.
24. In the land of the noble beaver where the rivers run wild with maple syrup, there lives a ne young gentleman named Kyle Fox. He's been developing nifty things for us in the moose-plagued landscape of Edmonton, Alberta since the beginning of 2011. At present he is working his sweet magic on our iPad app while wrestling bears, sipping ne Canadian whiskeys and tapping every last maple tree. We raise our glasses in celebration of his big day and thank him for being such a funny and valued member of Team Carbonmade. Happy birthday, eh?
25. Theme Park • A web-based tool for editing themes • Writes files (HTML/CSS/JS) directly to S3 • Asset bundling • Versioning • Deployments
26. ThemePark Online Editor Amazon S3 Used as CloudFront origin HTML CSS JS Liquid Files theme-<id>-<revision>.zip Publishing a theme
27. ThemePark Online Editor Amazon S3 Used as CloudFront origin HTML CSS JS Liquid Files theme-<id>-<revision>.zip Amazon SNS Theme Updated! Post to SNS
28. ThemePark Online Editor Amazon S3 Used as CloudFront origin HTML CSS JS Liquid Files theme-<id>-<revision>.zip Amazon SNS Theme Updated! Post to SNS Sinatra Sinatra Sinatra Theme Updated! Notify Subscribers
29. ThemePark Online Editor Amazon S3 Used as CloudFront origin HTML CSS JS Liquid Files theme-<id>-<revision>.zip Amazon SNS Theme Updated! Post to SNS Sinatra Sinatra Sinatra Theme Updated! Notify Subscribers Download templates
30. Publishing an SNS Message
31. Receiving an SNS Message
32. In conclusion Breaking up apps: • Makes testing & refactoring easier • Let’s you scale components individually & appropriately • Gives you freedom to drastically change how pieces work • Makes fault-tolerance & failover easier • AWS is a cheap & easy way to do this!