Fire It Up: How Empowered People Automated Provisioning in 6 Datacenters Across 4 Continents

This talk is a case study of a group of Rackers who automated network device provisioning by leveraging new and existing services.

Transcript

1. March 1, 2013 How Empowered People Automated Provisioning in 6 Datacenters Across 4 Continents Fire It Up Friday, March 1, 13
2. Me Friday, March 1, 13
3. Friday, March 1, 13
4. Friday, March 1, 13
5. Friday, March 1, 13
6. Friday, March 1, 13
7. even the cloud runs on hardware Friday, March 1, 13
8. trade-offs Friday, March 1, 13
9. SOME MEN JUST WANT TO WATCH THE WORLD BURN Friday, March 1, 13
10. Firewalls and Load Balancers = Devices Friday, March 1, 13
11. OS customization files = configs Friday, March 1, 13
12. Friday, March 1, 13
13. Friday, March 1, 13
14. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Provisioning Through Time 14 Friday, March 1, 13
15. Every technology goes through three stages: Friday, March 1, 13
16. first a crudely simple and quite unsatisfactory gadget; Friday, March 1, 13
17. second, an enormously complicated group of gadgets designed to overcome the shortcomings of the original and achieving thereby somewhat satisfactory performance through extremely complex compromise; Friday, March 1, 13
18. third, a final proper design therefrom. Friday, March 1, 13
19. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 19 Friday, March 1, 13
20. Friday, March 1, 13
21. what about that web app thingy people used to manually generate configs of dubious quality? Friday, March 1, 13
22. Rule of Separation Friday, March 1, 13
23. mechanism Friday, March 1, 13
24. policy Friday, March 1, 13
25. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Extracting A Service 25 Friday, March 1, 13
26. be your own client first Friday, March 1, 13
27. remain framework agnostic at the domain level Friday, March 1, 13
28. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Lack of Ubiquitous Language 28 Friday, March 1, 13
29. Friday, March 1, 13
30. Automating a process is just as much about organizational change as technical change Friday, March 1, 13
31. Friday, March 1, 13
32. Friday, March 1, 13
33. /swimlanes !"" deployment.rb !"" discovery.rb !"" network.rb #"" preparation.rb Friday, March 1, 13
34. class Swimlanes::Discovery < Swimlane # ... omitted ... def perform DeviceDiscovery.run(device) rescue => error @errors << error raise error ensure Orchestration.notify(device, self) end end Friday, March 1, 13
35. •learning together •easier troubleshooting Friday, March 1, 13
36. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Inflexible Consumers 36 Friday, March 1, 13
37. some API consumers make their own tradeoffs which limit their flexibility Friday, March 1, 13
38. POST /devices/12345/swimlanes/discovery Friday, March 1, 13
39. POST /devices/123456/kicks { “step”: “1 - Discovery” } Friday, March 1, 13
40. class KicksController < ApplicationController # ... omitted ... def create @kick_process = KickProcess.start(@device, params[:step]) respond_with(@kick_process) end end Friday, March 1, 13
41. class KickProcess STEPS = { ‘1 - Discovery’ => [ Swimlanes::Discovery ] } def self.start(device, step) kick_process = new(device, step) kick_process.start end def start STEPS[step].each do |action| action.perform(device) end self end end Friday, March 1, 13
42. •maintains flexibility •isolates inflexibility Friday, March 1, 13
43. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Coding Bottlenecks 43 Friday, March 1, 13
44. service objects Friday, March 1, 13
45. increased surface area Friday, March 1, 13
46. class DeviceDiscovery def self.run(device) device_discovery = new(device) device_discovery.run end def run switch = FE::Switch.discover(device) mac = switch.mac_address. select { |m| m == device.mac_address }.first FE::MacAddress.update(mac, device) end end Friday, March 1, 13
47. class DeviceDiscovery def self.run(device) device_discovery = new(device) device_discovery.run end def run switch = Switch.discover(device) mac = MacAddress.discover(switch) MacAddressCorrelation. create(device, mac) end end Friday, March 1, 13
48. easier troubleshooting Friday, March 1, 13
49. •increased surface area •easier composition •troubleshooting speed Friday, March 1, 13
50. trade-offs Friday, March 1, 13
51. https://twitter.com/plibin/status/106078180976238592 Friday, March 1, 13
52. BOOM. Value added. Friday, March 1, 13
53. RACKSPACE® HOSTING | WWW.RACKSPACE.COM THANK YOU @jschairb joshua.schairbaum@gmail.com 53 Friday, March 1, 13