Constructing Open Source SDKs for Ops Teams with REST and GraphQL

Operations teams are the front line team that have to install, configure, manage, and maintain a complex set of infrastructure and services. Getting a solid, well-documented SDK into their hands can turn manual, painful processes into simple automation tasks. However, as many APIs are shifting from REST to GraphQL, new approaches to tooling are often required. In this session, you’ll hear about how a small team of engineers builds and maintains a collection of open source SDKs focused on operations teams and how they added support for GraphQL endpoints.

Transcript

1. Constructing Open Source SDKs
   for Ops Teams with REST and
   GraphQL
   Chris Wahl
   

   View Slide

2. Chris Wahl
   ❖ Chief Technologist @ Rubrik
   ❖ Author of Networking for
   VMware Administrators
   ❖ Open Source Enabler at Rubrik
   Build
   ❖ he/him
   Twitter: @ChrisWahl
   GitHub: chriswahl
   LinkedIn: /wahlchris
   Blog: Wahl Network
   

   View Slide

3. @ChrisWahl | #DevWeek2019 3
   https://twitter.com/AxolotlCure/status/1136284938830045184
   

   View Slide

4. This is a story about toil
   And a lot of learning through triumph and mistakes
   @ChrisWahl | #DevWeek2019 4
   

   View Slide

5. “
   ”
   The kind of work tied to running a
   production service that tends to be
   manual, repetitive, automatable,
   tactical, devoid of enduring value, and
   that scales linearly as a service grows
   - Toil
   @ChrisWahl | #DevWeek2019 5
   

   View Slide

6. @ChrisWahl | #DevWeek2019 6
   

   View Slide

7. Life of an operator
   • At the end of the release cycle
   • “Here’s a thing, make it work, keep it
   working”
   • Myriad of systems to understand and
   maintain while being short staffed
   @ChrisWahl | #DevWeek2019 7
   

   View Slide

8. “
   ”
   I need a one-liner or script to
   accomplish this task so I can copy and
   paste it into my environment, solve my
   problem, and get back to putting out a
   hundred other fires
   - Systems Administrators
   @ChrisWahl | #DevWeek2019 8
   

   View Slide

9. Abuse from Crude Tools
   Tools like AutoIt
   • Script GUI actions using a
   DSL
   • The ultimate “sad panda”
   @ChrisWahl | #DevWeek2019 9
   

   View Slide

10. Key Ingredients
    @ChrisWahl | #DevWeek2019 10
    RESTful
    API
    Operator
    Audience
    Free
    Time
    SDK
    

    View Slide

11. Initial Research
    • Our audience preferred
    Microsoft PowerShell
    • Auto generation of SDK was ugly
    • Our swagger specification was
    non-standard
    • Decided to craft a bespoke SDK
    @ChrisWahl | #DevWeek2019 11
    

    View Slide

12. The Mission
    • Give operators a familiar tool to manage our product and remove toil
    • Use my background as an operator to control the UX
    • Selfishly: Learn how to build an SDK
    @ChrisWahl | #DevWeek2019 12
    

    View Slide

13. Project Plan
    • Everything in GitHub as an open source project
    • MIT licensing (Legal )
    • One project per repository
    • Official product support for projects
    • Unit tests for new features
    • External CI: AppVeyor, Azure Pipelines
    • Internal CI: CircleCI
    • Integration of Jira and GitHub via Zapier
    @ChrisWahl | #DevWeek2019 13
    

    View Slide

14. People use this thing?
    The mysterious tale of unloved APIs
    @ChrisWahl | #DevWeek2019 14
    

    View Slide

15. Our API’s Original Purpose
    • Distributed systems to chat with each other
    • Supply the GUI with an interface
    @ChrisWahl | #DevWeek2019 15
    me
    

    View Slide

16. This created friction
    • There were no API versions
    • Breaking changes were normal
    • Standards for model, params, enums, etc. did not exist
    • The product surface area was rapidly expanding
    @ChrisWahl | #DevWeek2019 16
    

    View Slide

17. @ChrisWahl | #DevWeek2019 17
    

    View Slide

18. @ChrisWahl | #DevWeek2019 18
    

    View Slide

19. We Made Versions!
    • Internal
    • meant for testing and developing new
    features and for providing command and
    control endpoints for the software itself.
    • Versioned (Vn)
    • meant for public consumption with a
    declaration on versioning, deprecation,
    and when breaking changes would be
    introduced.
    @ChrisWahl | #DevWeek2019 19
    

    View Slide

20. “
    ”
    API versioning does not prevent
    breaking changes. It just helps
    control when, where, and how the
    break occurs. Someone must still
    update their code.
    - Me
    @ChrisWahl | #DevWeek2019 20
    

    View Slide

21. More Cleanup
    • Placed major integrations at the parent (root) level
    • Leveraged HTTP methods to simplify workflows
    • Used Boolean field naming conventions
    @ChrisWahl | #DevWeek2019 21
    Ugly: POST to “/add_node” and “/remove_node/{id}”
    Pretty: POST to “/node” and DELETE to “/node/{id}”
    Start with ‘has’, ‘is’ or ‘should’ to make it clear that it is a Boolean field
    Examples: ‘hasRootAccess’, ‘isAdmin’ and ‘shouldDoSomething’
    

    View Slide

22. “
    ”
    The sooner you start to code,
    the longer the program will
    take.
    - Roy Carlson
    @ChrisWahl | #DevWeek2019 22
    

    View Slide

23. Internal Became the
    Hypnotoad
    • No incentives for versioning
    • Over 95% of the API resided in Internal
    @ChrisWahl | #DevWeek2019 23
    

    View Slide

24. The Universal Solvent
    Embracing our audience further
    @ChrisWahl | #DevWeek2019 24
    

    View Slide

25. Too Much Complexity
    • Each function with the SDK was a closed loop
    • The community found it too difficult to contribute
    • A new architecture was needed
    @ChrisWahl | #DevWeek2019 25
    

    View Slide

26. SDK Design Goal
    API File
    • Gather information for each
    supported endpoint
    • Supply the SDK with methods,
    params, status codes, etc.
    • Version the data for backwards
    compatibility
    Generic Functions
    • Functions look at the API File to
    understand their purpose
    • Functions can alter their state
    based on the target product
    version
    @ChrisWahl | #DevWeek2019 26
    

    View Slide

27. @ChrisWahl | #DevWeek2019 27
    Product versions 1.0+
    Product versions 5.0+
    

    View Slide

28. View Slide

29. View Slide

30. Enablement and
    Communication
    Too focused on the technology
    Not enough focus on the hygiene
    Lots of questions from our customers
    General fear of GitHub and coding
    More was needed
    @ChrisWahl | #DevWeek2019 30
    

    View Slide

31. View Slide

32. Choose Your Own Adventure
    

    View Slide

33. Educational Workshops for Operators
    

    View Slide

34. Communication Efforts
    @ChrisWahl | #DevWeek2019 34
    The rules of versioning and
    deprecation.
    Future deprecation of
    endpoints / resources.
    New or updated endpoints
    / resources.
    

    View Slide

35. And then GraphQL appeared
    There goes the neighborhood
    @ChrisWahl | #DevWeek2019 35
    

    View Slide

36. @ChrisWahl | #DevWeek2019 36
    

    View Slide

37. “
    ”
    You haven't mastered a tool
    until you understand when it
    should not be used.
    - Kelsey Hightower
    @ChrisWahl | #DevWeek2019 37
    

    View Slide

38. Initial Research in 2017
    • Dramatic speed improvements for the GUI
    • As more objects are added, REST continues to fall behind
    • Simple to query all objects and use cursor / pagination
    • More flexibility with our returned values
    @ChrisWahl | #DevWeek2019 38
    Stress tested load times
    95th percentile load times with GraphQL: 3.256 seconds
    95th percentile load times with REST: 6.619 seconds
    

    View Slide

39. Since Then
    • Added GraphQL to our on-premises
    product.
    • Reporting
    • Dashboards
    • Various other components
    • Constructed a SaaS platform with
    GraphQL as the standard API
    • Started from scratch
    • Using what we learned
    • Lots of tweaking
    @ChrisWahl | #DevWeek2019 39
    

    View Slide

40. Challenges
    • Schema is in flux
    • There are no versions
    • Documentation holy wars
    • We’re all still learning GraphQL
    • Graph-Que-What?
    @ChrisWahl | #DevWeek2019 40
    

    View Slide

41. Current State
    • Schema tools (Voyager, GraphiQL) for visualization
    • Internal construction of new SDKs
    • Existing auth methods (e.g. tokens) are valid globally
    @ChrisWahl | #DevWeek2019 41
    Base platform will continue with REST and GraphQL
    SaaS platform will remain entirely GraphQL
    Using GitHub private repos for development
    

    View Slide

42. View Slide

43. SDK Development
    Let use cases drive stack-ranking
    Mimic a near-identical UX
    Educate and enable in parallel
    Invite early-adopters and give them checklists
    @ChrisWahl | #DevWeek2019 43
    

    View Slide

44. Takeaways
    @ChrisWahl | #DevWeek2019 44
    

    View Slide

45. If we could do it all over again
    • Increased collaboration with engineering and support
    • Create incentives to document and polish the API
    • Make documentation a top priority
    • Educate internal stakeholders on API usage
    • Bring operators into your SDK build process
    @ChrisWahl | #DevWeek2019 45
    Use cases, UX, testing, feedback
    

    View Slide

46. Thank you
    Twitter: @ChrisWahl
    GitHub: chriswahl
    

    View Slide