Fire It Up: Automated Provisioning

At Rackspace, every minute spent on a manual task is a minute less spent providing fanatical support to customers. It's vital that we automate manual work. This talk is a case study of a group of Rackers who automated network device provisioning by leveraging new and existing services. Automating provisioning processes that stretch across the physical world and multiple applications is challenging. Shipping features on a time table and keeping a maintainable codebase require making intelligent tradeoffs. We want to share with you some of our insights into making these necessary tradeoffs.

Transcript

1. July 19, 2013 How Empowered People Automated Provisioning in 6 Datacenters Across 4 Continents Fire It Up Saturday, July 20, 13
2. Saturday, July 20, 13
3. we’ve had it with patent trolls Saturday, July 20, 13
4. Saturday, July 20, 13
5. Saturday, July 20, 13
6. even the cloud runs on hardware Saturday, July 20, 13
7. Saturday, July 20, 13
8. net device automation is vendor specific Saturday, July 20, 13
9. what about software defined networks? Saturday, July 20, 13
10. Firewalls and Load Balancers = Devices Saturday, July 20, 13
11. OS customization files = configs Saturday, July 20, 13
12. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Provisioning Through Time 12 Saturday, July 20, 13
13. hand-passed configs Saturday, July 20, 13
14. web-based config file generator Saturday, July 20, 13
15. hands-free automated system to deploy and configure Saturday, July 20, 13
16. Saturday, July 20, 13
17. Saturday, July 20, 13
18. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Hurdles 18 Saturday, July 20, 13
19. did not understand the domain Saturday, July 20, 13
20. Saturday, July 20, 13
21. testing physical devices is unforgiving Saturday, July 20, 13
22. Saturday, July 20, 13
23. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Make Once Decisions 23 Saturday, July 20, 13
24. Rule of Separation Saturday, July 20, 13
25. mechanism Saturday, July 20, 13
26. policy Saturday, July 20, 13
27. Focused on organizational change Saturday, July 20, 13
28. Automating a process is just as much about organizational change as technical change Saturday, July 20, 13
29. Saturday, July 20, 13
30. establish ports and adapters Saturday, July 20, 13
31. give integrators a walking skeleton Saturday, July 20, 13
32. Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law. Saturday, July 20, 13
33. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Make Daily Decisions 33 Saturday, July 20, 13
34. naming consistency Saturday, July 20, 13
35. /swimlanes !"" deployment.rb !"" discovery.rb !"" network.rb #"" preparation.rb Saturday, July 20, 13
36. class  Swimlanes::Discovery  <  Swimlane    #  ...  omitted  ...      def  perform        DeviceDiscovery.run(device)    rescue  =>  error        @errors  <<  error        raise  error    ensure        Orchestration.notify(device,  self)    end end Saturday, July 20, 13
37. Saturday, July 20, 13
38. providing a generally- named API Saturday, July 20, 13
39. some API consumers make their own tradeoffs which limit their flexibility Saturday, July 20, 13
40. POST /devices/123456/kicks { “step”: “1 - Discovery” } Saturday, July 20, 13
41. class  KicksController  <  ApplicationController    #  ...  omitted  ...    def  create        @kick_process  =  KickProcess.start(@device,                                                                            params[:step])        respond_with(@kick_process)    end end Saturday, July 20, 13
42. class  KickProcess    STEPS  =  {        ‘1  -­‐  Discovery’  =>  [  Swimlanes::Discovery  ]      }      def  self.start(device,  step)        kick_process  =  new(device,  step)        kick_process.start    end      def  start        STEPS[step].each  do  |action|            action.perform(device)        end        self    end end Saturday, July 20, 13
43. service objects Saturday, July 20, 13
44. increased surface area Saturday, July 20, 13
45. class  DeviceDiscovery    def  self.run(device)        device_discovery  =  new(device)        device_discovery.run    end      def  run        switch  =  FE::Switch.discover(device)        mac        =  switch.mac_address.            select  {  |m|                  m  ==  device.mac_address              }.first        FE::MacAddress.update(mac,  device)    end end Saturday, July 20, 13
46. class  DeviceDiscovery    def  self.run(device)        device_discovery  =  new(device)        device_discovery.run    end      def  run        switch  =  Switch.discover(device)        mac        =  MacAddress.discover(switch)        MacAddressCorrelation.            create(device,  mac)    end end Saturday, July 20, 13
47. RACKSPACE® HOSTING | WWW.RACKSPACE.COM THANK YOU @jschairb joshua.schairbaum@gmail.com 47 Saturday, July 20, 13