Developing Exceptional Architecture Design with Open Source and DevOps

Developing Exceptional Architecture Design with Open Source and DevOps

The transition from long-lived models used for on-premises architecture towards a world of public cloud, serverless, and on-demand resources requires a good, hard look at how we construct conceptual and logical functional designs. Fortunately, many exceptional methodologies and workflows already exist in the open source world that we can heavily lean upon. Couple this with the face melting velocity experienced by those who pattern their release flows after the pillar characteristics of a DevOps world to start seeing a blending of on-premises and cloudy dimensions. In this talk, you’ll hear about current trends in various ecosystem industries, where the state of our art is headed, DevOps success stories, and what you can do to get started today!

80a34a84e6b006b603b1192a8af3b435?s=128

Chris Wahl

March 07, 2019
Tweet

Transcript

  1. 3.

    Agenda • What The Heck is Going On? • Learning

    from Open Source • Practice what you Preach • So, What Does This Look Like? • Interesting Use Cases! • Parting Thoughts
  2. 5.

    The Counter-Industrial Revolution “At the cusp of the 20th and

    21st centuries, intangible assets overtook tangible assets in the economy” • Computerized information • Innovative property • Economic competencies
  3. 6.

    The Counter-Industrial Revolution 1. The optimal scale for digital assets

    is global. 2. The home has re-emerged as a workplace for teleworkers. 3. The gig economy has sprung up at the expense of wage-based employment. 4. Many firms delay IPO, preferring to remain private because they regard venture capitalists as better positioned to understand the value of their intangible assets.
  4. 7.
  5. 11.
  6. 12.

    Open Source Projects are Hard It all stems from why,

    how, and what. • How do you get folks interested? • What carrots or sticks do you have available? • How many goats can you herd at once? • How do you scale? If you don’t believe in the mission, no one else will either.
  7. 14.

    Overview – 2011 Project • Managing a few thousand pizza

    boxes • Working on a very small team • Leaving behind cheeky MOTDs for the lulz
  8. 15.

    Challenges • Team of two • Customers: 500+ developers around

    the world • Environment would scale in really wonky ways, daily
  9. 16.
  10. 17.

    Looked for Inspiration Elsewhere • Found the OpenStack project •

    Figured Nova and Cinder would be interesting projects • Knew enough Python to get messy • Largely just interested in large scale development • How does the other side operate? • What lessons could I adapt to ops?
  11. 18.

    Learnings • My “two pizza team” was kind of nice

    • The grass wasn’t any greener, just a different shade of sad • DVCS is dope!
  12. 19.

    Learnings Operations often gets in the way • Sometimes for

    good – security, performance, capacity planning • Sometimes for bad – lack of understanding, silos, NIMBY
  13. 21.

    Takeaways Contacted the facilities team to get a seating chart

    • Found where the dev leaders sat • Went to lunch, talked about our challenges • Shared knowledge and roadmap to align efforts • Borrowed a dev to clean up my messy code + ServiceNow workflows I learned that communication is always* the problem
  14. 32.

    As It Applies to Rubrik • 100% distributed team, globally

    • Workflows, documentation, and pipeline-driven process drive most everything • Building “all the things” in two on-prem facilities and three clouds (Azure, AWS, GCP) in a variety of regions • Default = native tools > overlays • We have no dedicated “ops” people • The work is never done, but that’s OK
  15. 33.
  16. 34.

    We operate as engineers There are some folks who are

    pretty good at advocating for this: “Netflix’s engineering culture is predicated on Freedom & Responsibility, the idea that everyone (and every team) at Netflix is entrusted with a core responsibility and they are free to operate with freedom to satisfy their mission.” This is the premise behind contractual operations
  17. 35.
  18. 36.

    Contractual Operations Any system we create / operate is too

    complex for any one person or team to know all of it. Take API contracts as an example: • Machine readable definition of an API interface (e.g. Swagger). • Definition of the surface area of the resources that are available. • Describe, communicate, and collaborate around APIs. • These are often extremely abstract.
  19. 37.

    Contractual Operations: Requirements • Clearly define your inputs and outputs

    • Document the heck out of everything • Let people know when things change • Be creative within that realm • Build for scale • Unless it works idempotently, it’s wrong • Assume you will spontaneously combust at any moment • Embrace Service Ownership
  20. 39.

    Production Readiness ❑Security ❑Logging ❑Monitoring ❑Alerting ❑Observability (ephemeral) ❑Enabling Features

    ❑Testing ❑SLOs ❑Costs ❑Performance ❑Deployment / Build ❑Enablement
  21. 40.
  22. 41.
  23. 42.

    Musings on Building for Scale • Consider everything an artifact

    that needs a unique ID • Use declarative languages, DSLs, or native tooling when possible • Suggest changes in DVCS (Git), not by hand or dead tree (paper) • Style guides are dope • No tests = stop, just stop!
  24. 43.
  25. 44.

    Things My Team Likes • Azure DevOps is fairly solid

    as a build / release tool • Jira works for project management, Kanban boards, and getting the attention of other engineers ☺ • IBM RedHat CloudForms / ManageIQ (cloud management) • Terraform (config management; built on our Go SDK) • Ansible (config management; built on our Python SDK)
  26. 46.

    Thoughts • Communicate your plans • Find pain points to

    solve and innovation to copy • Design out in the open • Don’t consume anything that doesn’t have awesome APIs • Test everything • Assume good intent • Only punish those who build by hand (mistakes happen) • Version control everything • Pipeline everything • Test everything (!!!)