SOA Safari in the Amazon

Transcript

1. SOA Safari in the Amazon @bryanthompson Friday, April 5, 13
2. Friday, April 5, 13
3. BACKGROUND on BRYANTHOMPSON Friday, April 5, 13
4. • VP of Development, PaymentSpring • Ruby minimalist • Obvious Framework – co-creator Side projects: • bizcard.io, heydummy.io • Crontabulator.com • Milksteak – flat file cms • dozens of half-finished toys Friday, April 5, 13
5. • VP of Development, PaymentSpring • Ruby minimalist • Obvious Framework – co-creator Side projects: • bizcard.io, heydummy.io • Crontabulator.com • Milksteak – flat file cms • dozens of half-finished toys bryan thompson Friday, April 5, 13
6. Friday, April 5, 13
7. “the worst code that involves Rails I’ve ever seen” Friday, April 5, 13
8. “the worst code that involves Rails I’ve ever seen”- dhh Friday, April 5, 13
9. • VP of Development, PaymentSpring • Ruby minimalist • Obvious Framework – co-creator Side projects: • bizcard.io, heydummy.io • Crontabulator.com • Milksteak – flat file cms • dozens of half-finished toys bryan thompson Friday, April 5, 13
10. This talk is... not an intro to SOA Friday, April 5, 13
11. Friday, April 5, 13
12. Friday, April 5, 13
13. This talk is... not a design pattern talk Friday, April 5, 13
14. This talk is... not all about amazon Friday, April 5, 13
15. This talk is... about problems and solutions Friday, April 5, 13
16. We are building a payment gateway Needs to be: elastic, scalable, regionally fault tolerant, fast, and PCI level 1 compliant Reasons for SOA: Rewrite any service in any language at any time Scale any service independently PCI Requirement 2.2.1: Implement only one primary function per server our situation... Friday, April 5, 13
17. cloudy requirements • Our basic needs • PCI Compliant facility & infrastructure tools • Full control of networking • Full control of provisioning • High priorities • Autoscaling ability, even if we write it ourselves • no single-datacenter solutions Friday, April 5, 13
18. Requirement 6.1: “all system components and software are protected from known vulnerabilities by having the latest vendor supplied security patches installed.” Friday, April 5, 13
19. Requirement 6.1: “all system components and software are protected from known vulnerabilities by having the latest vendor supplied security patches installed.” Friday, April 5, 13
20. anything that helps manage the platform is in scope (landscape, cloud passage, rightscale, hosted CI, etc.) Friday, April 5, 13
21. every update of every gem is now in scope. Friday, April 5, 13
22. what the fuck is this new version doing? security, bugfix, feature, A SPELLING CORRECTION?!???? #flipsdesk Friday, April 5, 13
23. Update nothing. Throw it away and deploy fresh systems and code. Friday, April 5, 13
24. Problem: configuration files are readable Friday, April 5, 13
25. Encrypts/Decrypts files, messages, keys, etc. Includes gem & CLI tools for puppet and services Talks to our keyserver, does a lot of super weird things with passwords/keys. CRYPTAR . Friday, April 5, 13
26. Problem: keeping gem versions in sync Friday, April 5, 13
27. Environment Gem Separation • we use separate gemfiles • you could use bundler groups Reviewing gem updates • pull latest gems into development • diff lock file for version detail • check out code and diff between versions Issues • isn’t the swiftest process ever • changelogs – thank you to those who care geminator Friday, April 5, 13
28. Problem: keeping system updates in sync Friday, April 5, 13
29. Looks at all systems in production and grabs list of available system updates. Lets us assign risk rank quickly by also scraping CVE issues related to updates. Instances are replaced within a given timeframe based on the risk of the applicable updates. PATCHINATOR Friday, April 5, 13
30. continuous integration Friday, April 5, 13
31. continuous integration BUILDATION! Friday, April 5, 13
32. Core CI is powered by Jenkins CI environment is rebuilt between tests We run CI every time: • we apply system updates • we apply gem updates • we tag code for release • we modify a puppet manifest • change any firewall rules always integrating Friday, April 5, 13
33. Problem: rapidly deploying code & systems Friday, April 5, 13
34. Friday, April 5, 13
35. Friday, April 5, 13
36. Handles System & Code Deployments Lets us tag instances with useful version information. 3 stages of deployment: • Stage 0 AMI comes from Ubuntu • Stage 1 AMI all our puppet manifests and updates • Stage 2 AMI production-ready w/ our code Can deploy new versions of everything within 10 minutes. deployinator Friday, April 5, 13
37. Friday, April 5, 13
38. This means we are killing 62 servers permanently Friday, April 5, 13
39. A few oddities... • AWS Console isn’t perfect • Native AWS REST API vs aws-sdk gem • hack around with API • then write scripts • isn’t always the swiftest • takes multiple calls to jump down the tree • Load Balancers • Re-register instances when cold • Randomly takes “forever” Friday, April 5, 13
40. a few things you can do right now • start thinking about disposability in your env • use vagrant to learn some automation • find your puppet/chef/vlad deployment groove • write your own automation scripts • build AMIs and deploy from them • keep your AMIs updated Friday, April 5, 13
41. Thanks for listening! @bryanthompson Friday, April 5, 13
42. notes http://www.infoq.com/presentations/Some-REST-Design-Patterns http://www.infoq.com/articles/SOA-Design-Patterns-Thomas-Erl-2 http://mtnwestrubyconf.org/2011/sessions http://confreaks.com/videos/1589-railsberry2012-take-the-pita-out- of-soa-building-lean-service-oriented-architectures https://speakerdeck.com/brynary Friday, April 5, 13