Creating a fast Kubernetes Development Workflow

Kubernetes is very powerful in orchestrating where your services are running, but building docker images, writing and updating all the necessary YAML files and applying them to your cluster can still become an annoying maintenance overhead. In this talk you will learn how to create a fast, low friction development workflow with Kubernetes and tools like Telepresence and Forge which enables you to quickly and safely build, test and deploy your services.

Transcript

1. @BastianHofmann
   Creating a fast
   Kubernetes Development Workflow
   Bastian Hofmann
   

   View Slide

2. View Slide

3. Container orchestration platform
   

   View Slide

4. Deploy, run and scale your services
   in isolated containers
   

   View Slide

5. No vendor lock in
   

   View Slide

6. Runs on
   

   View Slide

7. Your laptop
   

   View Slide

8. View Slide

9. Bare metal
   

   View Slide

10. Cloud Providers
    

    View Slide

11. And if you don't want to install and
    maintain Kubernetes yourself
    

    View Slide

12. Managed Kubernetes
    

    View Slide

13. View Slide

14. Standardized APIs
    

    View Slide

15. It works the same everywhere*
    

    View Slide

16. It works the same everywhere*
    *mostly
    

    View Slide

17. This talk is about
    how to use Kubernetes
    

    View Slide

18. Not only for production workloads
    

    View Slide

19. But in your development workflows
    

    View Slide

20. Agenda
    

    View Slide

21. Deployment of a micro-service
    application
    

    View Slide

22. Some tools to help with local
    development of this application on
    Kubernetes
    

    View Slide

23. Let's have a look at the sample
    application
    

    View Slide

24. View Slide

25. OpenStack Cloud LoadBalancer
    NGINX Ingress Controller NGINX Ingress Controller NGINX Ingress Controller
    web-application web-application
    MySQL Master
    MySQL Slave
    quote-svc
    quote-svc
    hello-svc
    hello-svc
    

    View Slide

26. external-dns to create DNS entries
    automatically
    

    View Slide

27. cert-manager to retrieve
    Let's Encrypt certificates
    automatically
    

    View Slide

28. Database is managed by an Operator
    

    View Slide

29. MySQL Operator MySQLCluster
    MySQL pods
    MySQL statefulset
    Kubernetes controller manager
    Discovers
    Creates
    Creates
    Discovers
    Monitors and manages
    

    View Slide

30. LinkerD as a Service Mesh for
    Telemetry
    

    View Slide

31. View Slide

32. If you are interested in the code and
    how to set it up:
    https:/
    /github.com/syseleven/
    golem-workshop
    

    View Slide

33. Demo
    

    View Slide

34. Writing this YAML files is tedious
    

    View Slide

35. YAML files are tied to a specific
    version and a specific environment
    

    View Slide

36. Production
    

    View Slide

37. Staging
    

    View Slide

38. Development
    

    View Slide

39. Per Development team
    

    View Slide

40. Per branch
    

    View Slide

41. Per developer
    

    View Slide

42. We need to maintain multiple, very
    similar YAML files with slightly
    different versions and configuration
    

    View Slide

43. "Templating"
    

    View Slide

44. Great tools because of standardized
    Kubernetes API
    

    View Slide

45. Helm
    

    View Slide

46. View Slide

47. Allows to install applications
    

    View Slide

48. So called "charts"
    

    View Slide

49. $ helm install stable/wordpress \
    --name my-blog \
    --namespace blog
    

    View Slide

50. Charts can depend on other charts
    

    View Slide

51. Multiple deployments of one chart
    possible
    

    View Slide

52. Different release names
    

    View Slide

53. Different namespaces
    

    View Slide

54. Configuration with values
    

    View Slide

55. View Slide

56. $ helm install stable/wordpress \
    --name my-blog \
    --namespace blog \
    -f my-config-values.yaml
    

    View Slide

57. Writing your own charts is fairly
    easy
    

    View Slide

58. Scaffolding to get started
    

    View Slide

59. $ helm create quote-svc
    

    View Slide

60. $ helm install ./quote-svc \
    --namespace dev-bastian \
    --name dev-bastian-quote-svc \
    --values dev.yaml --values bastian.yaml
    

    View Slide

61. Demo
    

    View Slide

62. Still, for development:
    

    View Slide

63. Make a code change
    

    View Slide

64. Build docker image
    

    View Slide

65. Push docker image
    

    View Slide

66. Run helm install/upgrade with new
    image version
    

    View Slide

67. Can this be quicker?
    

    View Slide

68. Run everything locally
    

    View Slide

69. docker-compose
    

    View Slide

70. Duplication of the definition of how
    to run a container
    

    View Slide

71. Inconsistencies
    

    View Slide

72. If you have a lot of services, you have
    to run a lot locally
    

    View Slide

73. Some services locally, some remote
    

    View Slide

74. Service Discovery
    

    View Slide

75. Not every service is exposed to the
    Internet
    

    View Slide

76. Shared resources with other
    developers?
    

    View Slide

77. Other options?
    

    View Slide

78. Tilt
    

    View Slide

79. $ tilt up
    

    View Slide

80. Watches for code changes
    

    View Slide

81. Rebuilds docker image
    

    View Slide

82. Deploys to Kubernetes
    

    View Slide

83. Sets up port-forwarding
    

    View Slide

84. Can sync changed files directly into a
    running container
    

    View Slide

85. Demo
    

    View Slide

86. Another approach
    

    View Slide

87. View Slide

88. Creates a two-way proxy between
    the Kubernetes cluster and you
    

    View Slide

89. $ telepresence
    T: Starting proxy with method 'vpn-tcp'...
    @fhgbvx65xg|bash-3.2$ curl http://quote-svc/quote | jq '.'
    [
    {
    "ID": 503,
    "title": "stefan sagmeister",
    "content": "...\n",
    "link": "https://quotesondesign.com/stefan-
    sagmeister-2/"
    }
    ]
    

    View Slide

90. Swap a running deployment in the
    cluster with a local process
    

    View Slide

91. ... or a locally running docker
    container
    

    View Slide

92. $ telepresence --swap-deployment quote-svc --namespace
    dev-flow-demo --expose 3000 --run npm run debug
    T: Starting proxy with method 'vpn-tcp',...
    T: Forwarding remote port 3000 to local port 3000....
    > quote-svc@1.0.0 debug /Users/bhofmann/forge_test/quote-
    svc
    > nodemon --inspect quote-svc.js
    [nodemon] watching: *.*
    [nodemon] starting `node --inspect quote-svc.js`
    Debugger listening on ws://127.0.0.1:9229/83aa27ac-
    d879-4b50-a228-440354cca791
    quote svc listening on port 3000!
    

    View Slide

93. Demo
    

    View Slide

94. Summary
    

    View Slide

95. Powerful
    

    View Slide

96. Great tooling because of common
    APIs
    

    View Slide

97. Especially great if you have multiple
    services and don't want to run
    everything locally
    

    View Slide

98. I just picked helm, tilt and
    telepresence. There is more for
    different use-cases.
    

    View Slide

99. mail@bastianhofmann.de
    https:/
    /twitter.com/BastianHofmann
    http:/
    /speakerdeck.com/u/bastianhofmann
    https:/
    /github.com/syseleven/golem-workshop
    

    View Slide