Horizontal Scaling in the Cloud

An overview of how FeedMagnet uses Ansible to accomplish horizontal scaling in the cloud.

Transcript

1. Horizontal Scaling
 in the Cloud (with Ansible, of course!) ! Luke Sneeringer (@lukesneeringer) FeedMagnet ! Slides available: lukesn.me/ansibleconf2014
2. • I'm the lead developer at FeedMagnet. • We do: Social curation.
 We take information from places like Twitter, Facebook, Instagram, etc. and allow companies to filter and redisplay it for their purposes. lukesn.me/ansibleconf2014 @lukesneeringer
3. A History • Started out as a self-service organization. • Everyone's data lived together in the same database. • The scale of social data for non-trivial customers is huge. • ...and a lot of it is garbage, but we can't get rid of it. lukesn.me/ansibleconf2014 @lukesneeringer
4. Bad Neighbor Problem • One client's pulling in more data than we could handle broke all the things™. • Too many web requests from someone with an active event...broke all the things™. • An effective sales team eventually broke, you guessed it, all the things™. @lukesneeringer lukesn.me/ansibleconf2014
8. The Solution • We were (and still are) an extremely small organization. • No necessity for multi-tenancy. • Single-tenancy is the solution. @lukesneeringer lukesn.me/ansibleconf2014
9. Single-Tenancy • It's much more scalable as we get more business. • Completely avoids problems with bleed-over and bad neighbors. • It's cheap! @lukesneeringer lukesn.me/ansibleconf2014
10. Single-Tenancy • It's cheap in labor, because it brings down the scalability requirements for our application by orders of magnitude. • It's cheap in hardware, because it brings down the scalability requirements for our application by orders of magnitude. @lukesneeringer lukesn.me/ansibleconf2014
12. The Structure • The first pass: two rigid roles. • deployment • utility • Slight size differentiation, but that's all. • We'll sell you any color, as long as it's black. @lukesneeringer lukesn.me/ansibleconf2014
13. The Structure • Adding nuance: • Splitting roles across machines: • Stand-alone database server. • Splitting machines within roles: • Multiple web servers. • Multiple worker bees. @lukesneeringer lukesn.me/ansibleconf2014
14. But wait! I haven't mentioned Ansible yet! @lukesneeringer lukesn.me/ansibleconf2014
16. Single-Tenancy • But single-tenancy is easy in Ansible. • Define roles, and assign roles to servers as required. @lukesneeringer lukesn.me/ansibleconf2014
17. Single-Tenancy • These are our current roles: • web • web load balancer • database • worker • util @lukesneeringer lukesn.me/ansibleconf2014
18. The Plot Twist We didn't use Ansible to start! @lukesneeringer lukesn.me/ansibleconf2014
20. We converted. @lukesneeringer lukesn.me/ansibleconf2014
21. Multi-Tenancy Returns • Not everything should be single-tenant, even in a largely single-tenant application. • One example for us: authentication. • This is where Ansible really starts to shine. @lukesneeringer lukesn.me/ansibleconf2014
22. The Challenge • Don't repeat yourself. • In Ansible, that probably means that you should be writing plugins. • If you don't have any custom Ansible plugins, consider writing some. @lukesneeringer lukesn.me/ansibleconf2014
24. Authentication • We use multi-tenant auth (single sign-on), even though most of our application is single-tenant. • Solves a business need: one client with more than one instance. • How is this related to server orchestration? @lukesneeringer lukesn.me/ansibleconf2014
25. Authentication • The other half of user management is handling internal users. • Internal users may have SSH access, files to track, etc. • How do we have one concept of application users and system users? @lukesneeringer lukesn.me/ansibleconf2014
27. Authentication • Lookup plugins (like other plugins) are easy to write in Ansible. • The power of lookup plugins is (partially) in their ability to allow you to not repeat yourself. @lukesneeringer lukesn.me/ansibleconf2014
28. Authentication • How we approached this: admin users may also have an SSH user record, the latter of which stores data specific to system users (authorized keys, for instance). • This also supports file management. @lukesneeringer lukesn.me/ansibleconf2014
29. File Management • We keep our files in a GitHub repository, one branch per person. • GitHub sends push requests to Wardrobe to put the contents of those files in the database. • ...and a 38-line Ansible lookup plugin gets them from the database. @lukesneeringer lukesn.me/ansibleconf2014
31. Authentication • Here's why this is so awesome: • All our user management, for anything a user can be, is in one place. • If an employee quits, we turn them off in one place. • If an employee is promoted, we change their data in one place. @lukesneeringer lukesn.me/ansibleconf2014
32. ...and Ansible makes it happen everywhere. @lukesneeringer lukesn.me/ansibleconf2014
33. But that's not all! • The same system that is the definitive source for users is also the definitive source for servers. • Ansible's pluggable inventory system makes this possible. @lukesneeringer lukesn.me/ansibleconf2014
34. Conclusion • This just scratches the surface of what we do with Ansible right now. • I want to encourage you to be thinking about how you can avoid copying data. • Hook Ansible up to your definitive data source. @lukesneeringer lukesn.me/ansibleconf2014