Developing Exceptional Architecture Design with Open Source and DevOps

The transition from long-lived models used for on-premises architecture towards a world of public cloud, serverless, and on-demand resources requires a good, hard look at how we construct conceptual and logical functional designs. Fortunately, many exceptional methodologies and workflows already exist in the open source world that we can heavily lean upon. Couple this with the face melting velocity experienced by those who pattern their release flows after the pillar characteristics of a DevOps world to start seeing a blending of on-premises and cloudy dimensions. In this talk, you’ll hear about current trends in various ecosystem industries, where the state of our art is headed, DevOps success stories, and what you can do to get started today!

Transcript

1. Developing Exceptional Architecture Design with Open Source and DevOps
2. Chris Wahl Chief Technologist, Rubrik chris.wahl@rubrik.com chriswahl @ChrisWahl
3. Agenda • What The Heck is Going On? • Learning from Open Source • Practice what you Preach • So, What Does This Look Like? • Interesting Use Cases! • Parting Thoughts
4. What The Heck is Going On? A fancy version of “Start with Why”
5. The Counter-Industrial Revolution “At the cusp of the 20th and 21st centuries, intangible assets overtook tangible assets in the economy” • Computerized information • Innovative property • Economic competencies
6. The Counter-Industrial Revolution 1. The optimal scale for digital assets is global. 2. The home has re-emerged as a workplace for teleworkers. 3. The gig economy has sprung up at the expense of wage-based employment. 4. Many firms delay IPO, preferring to remain private because they regard venture capitalists as better positioned to understand the value of their intangible assets.
7. The Counter-Industrial Revolution Spiral Motion Linear building Batch time Physical I/O Circular evolution Real-time Data (& memes) I/O
8. With that said …
9. Link: https://twitter.com/kevinbehr/status/1098707964721094656
10. Link: https://twitter.com/ChrisWahl/status/1080986977485406208
11. Learning from Open Source A sprinkle of development, a dash of chaos, and a pinch of creativity
12. Open Source Projects are Hard It all stems from why, how, and what. • How do you get folks interested? • What carrots or sticks do you have available? • How many goats can you herd at once? • How do you scale? If you don’t believe in the mission, no one else will either.
13. That should sound a bit familiar
14. Overview – 2011 Project • Managing a few thousand pizza boxes • Working on a very small team • Leaving behind cheeky MOTDs for the lulz
15. Challenges • Team of two • Customers: 500+ developers around the world • Environment would scale in really wonky ways, daily
17. Looked for Inspiration Elsewhere • Found the OpenStack project • Figured Nova and Cinder would be interesting projects • Knew enough Python to get messy • Largely just interested in large scale development • How does the other side operate? • What lessons could I adapt to ops?
18. Learnings • My “two pizza team” was kind of nice • The grass wasn’t any greener, just a different shade of sad • DVCS is dope!
19. Learnings Operations often gets in the way • Sometimes for good – security, performance, capacity planning • Sometimes for bad – lack of understanding, silos, NIMBY
20. Takeaways • We adopted Git as our DVCS • Peer reviews = death to CABs!
21. Takeaways Contacted the facilities team to get a seating chart • Found where the dev leaders sat • Went to lunch, talked about our challenges • Shared knowledge and roadmap to align efforts • Borrowed a dev to clean up my messy code + ServiceNow workflows I learned that communication is always* the problem
22. *but sometimes the problem is just DNS
23. Practice what you Preach A different approach to operating a team
24. Some call this DevOps ¯\_(ツ)_/¯
25. Link: https://twitter.com/ScribblingOn/status/1101548686344114176
26. Achievement Unlocked: Developer
27. Link: https://twitter.com/srockets/status/1097642296261074945
28. Technical Debt is a tradeoff for speed Eventually, you need to pay it down
29. Link: https://twitter.com/tactical_intel/status/1101717575388487680
30. Infra vendors
31. So, What Does This World Look Like? How the sausage is CI/CD pipelined
32. As It Applies to Rubrik • 100% distributed team, globally • Workflows, documentation, and pipeline-driven process drive most everything • Building “all the things” in two on-prem facilities and three clouds (Azure, AWS, GCP) in a variety of regions • Default = native tools > overlays • We have no dedicated “ops” people • The work is never done, but that’s OK
34. We operate as engineers There are some folks who are pretty good at advocating for this: “Netflix’s engineering culture is predicated on Freedom & Responsibility, the idea that everyone (and every team) at Netflix is entrusted with a core responsibility and they are free to operate with freedom to satisfy their mission.” This is the premise behind contractual operations
35. My team’s skill requirements: Learn a language Learn Git Learn REST (and GraphQL) Act like an adult
36. Contractual Operations Any system we create / operate is too complex for any one person or team to know all of it. Take API contracts as an example: • Machine readable definition of an API interface (e.g. Swagger). • Definition of the surface area of the resources that are available. • Describe, communicate, and collaborate around APIs. • These are often extremely abstract.
37. Contractual Operations: Requirements • Clearly define your inputs and outputs • Document the heck out of everything • Let people know when things change • Be creative within that realm • Build for scale • Unless it works idempotently, it’s wrong • Assume you will spontaneously combust at any moment • Embrace Service Ownership
38. This works regardless of locality or topology
39. Production Readiness ❑Security ❑Logging ❑Monitoring ❑Alerting ❑Observability (ephemeral) ❑Enabling Features ❑Testing ❑SLOs ❑Costs ❑Performance ❑Deployment / Build ❑Enablement
42. Musings on Building for Scale • Consider everything an artifact that needs a unique ID • Use declarative languages, DSLs, or native tooling when possible • Suggest changes in DVCS (Git), not by hand or dead tree (paper) • Style guides are dope • No tests = stop, just stop!
44. Things My Team Likes • Azure DevOps is fairly solid as a build / release tool • Jira works for project management, Kanban boards, and getting the attention of other engineers ☺ • IBM RedHat CloudForms / ManageIQ (cloud management) • Terraform (config management; built on our Go SDK) • Ansible (config management; built on our Python SDK)
45. In Conclusion
46. Thoughts • Communicate your plans • Find pain points to solve and innovation to copy • Design out in the open • Don’t consume anything that doesn’t have awesome APIs • Test everything • Assume good intent • Only punish those who build by hand (mistakes happen) • Version control everything • Pipeline everything • Test everything (!!!)
47. Thank you, wonderful humans! chris.wahl@rubrik.com chriswahl @ChrisWahl