Build It Yourself: How a Small Team Deployed OpenStack

Inside Puppet Labs, the SysOps team has been fighting year after year to rip up our entrenched virtualization system. Well, it is finally happening. Comprised of a very small two person team we deployed our first production grade OpenStack cloud and delivered it as a research platform to the entire company. Being what we are, a company that writes automation software and a small team with goals of using a common set of Unix/Linux tools; we decided to go it our own and build the best we could without picking up any downstream installers or commercially curated automation.

Video available at: https://www.youtube.com/watch?v=7pqKNwiGtx4

Transcript

1. Build It Yourself How a Small Team Deployed OpenStack Build It Yourself: How a Small Team Deployed OpenStack
2. Cody Herriges Principal SysOps Engineer @ Puppet Build It Yourself: How a Small Team Deployed OpenStack
3. The shortest path to better software.
4. Team Identity Developer minded automation enthusiasts enabling a more effective work environment 4
5. HACKER “The hacker culture is a subculture of individuals who enjoy the intellectual challenge of creatively overcoming and circumventing limitations of systems to achieve novel and clever outcomes” - Wikipedia 5
6. Tendencies Very wary of implementing anything that isn’t at least majority open source Not afraid to dispose of large chunks of infrastructure off cliffs Always leaning towards build over buy Generalists with focused interests Skeptical Agnostic Build It Yourself: How a Small Team Deployed OpenStack 6
7. Failures along the way Previous attempts • AWS • Vanilla vSphere • Pre-provisioned “tenants” and CLI shims on top of vSphere • RESTful API to abstract away vSphere’s SOAP API Unlike the Portland offices’ enjoyable and successful arcade we’ve made some arguably misguided steps in our way to providing self-service infrastructure that people find a pleasure to use. Build It Yourself: How a Small Team Deployed OpenStack 7
8. SLICE Self-Service Low Impedance Compute Environment 8
9. Goals • Safe isolated environment to try new things • Low friction for end user onramp to value • FAST! • Flexible enough to handle new ways of thinking • Automatable • Open source stack of industry standard tools for troubleshooting and management Build It Yourself: How a Small Team Deployed OpenStack 9
10. SLICE by the numbers Open Source 100% open source software and hardware vendor agnostic People 1 full time and two floaters Automation 4000 lines of freshly written automation and 10 upstream community modules Packaging 37 rebuilt, built, or backported and one 1 more incoming A lot went into this project that spanned 20 months from official start to production launch Build It Yourself: How a Small Team Deployed OpenStack 10
11. Issues Knowledge OpenStack has bugs Obscurely documented workarounds We built it more HA than we are use to Our first full Python based piece of software in production Build It Yourself: How a Small Team Deployed OpenStack 11
12. Knowledge OpenStack is dense and we believe understanding a platform before automating it 12
13. Bugs Most are actually already fixed, you just have to go get the code yourself…but then some aren’t. http://herrig.es/os-ssl-issue I am unable to find evidence this is fixed in Neutron 13
14. Workarounds Most a result of previous mentioned issue with SSL termination 14
15. HA New monitoring needs and having a dependence on Puppet always working 15
16. Our first python app Quickly learned the multiple ways our Puppet was installing Python 16
17. Make simple choices 17
18. Agnosticism Dogmatism is your enemy… 18
19. Learn from the ecosystem Originally we thought we’d do something different 19
20. Only deploy what you NEED You probably don’t need an SDN and we’re yet to deploy Cinder 20
21. Active/Active HA Really is easier to maintain and less prone to complete failure 21
22. Mature automation Even we could have gone it alone or off the rails…but we didn’t 22
23. Puppet OpenStack • Robust CI matrix • 37+ active repositories • Commercially downstreamed by Mirantis and RedHat • Active operator community: TWC, Puppet, HPE, UFCG, and many more An official OpenStack Big Tent project Established circa May 25, 2011 Build It Yourself: How a Small Team Deployed OpenStack 23
24. Skills to have or obtain 24
25. Python Expertise not required 25
26. Packaging Packaging RPM, DEB, or Docker doesn’t matter, just be confident 26
27. Packet tracing and dumping Even without SDN there are a lot of paths a packet can take 27
28. SQL Not everything is available via public API 28
29. OpenStack Basic understanding before you automate and find your self some early adopters 29
30. The End