Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Docker & Kubernetes Practices for Java Developers

Docker & Kubernetes Practices for Java Developers

A collection of practices, tips & tricks, for constructing Docker Containers and using Kubernetes for Java developers.

This slide is based on materials from:

Docker Tips and Tricks for Java Developers (Ray Tsang @saturnism)

Kubernetes Best Practices (Sandeep Dinesh @sandeepdinesh)

Kubernetes Security Best Practices (Ian Lewis @ianmlewis)


Ray Tsang

April 26, 2018


  1. for Java Developers Docker & Kubernetes Practices

  2. 2 @saturnism @googlecloud @kubernetesio Ray Tsang Developer Advocate Google Cloud

    Platform Spring Cloud GCP cloud.spring.io/spring-cloud-gcp/ @saturnism | +RayTsang
  3. 3 @saturnism @googlecloud @kubernetesio Ray Tsang Developer Architect Traveler Photographer

  4. 4 @saturnism @googlecloud @kubernetesio 4 A Short Recap

  5. 5 @saturnism @googlecloud @kubernetesio apiVersion: extensions/v1beta1 kind: Deployment metadata: name:

    work-server-v1 ... spec: replicas: 2 template: ... spec: containers: - name: work-server image: saturnism/work-server-istio:v1
  6. 6 @saturnism @googlecloud @istiomesh @kubernetesio web browsers Scheduler kubectl web

    browsers scheduler Kubelet Kubelet Kubelet Kubelet Config file Kubernetes Master Container Image
  7. 7 @saturnism @googlecloud @istiomesh @kubernetesio Let's see it...

  8. 8 @saturnism @googlecloud @istiomesh @kubernetesio Let's see some practices... Based

    on materials from: • Docker Tips and Tricks for Java Developers (Ray Tsang) • Kubernetes Best Practices (Sandeep Dinesh) • Kubernetes Security Best Practices (Ian Lewis)
  9. 9 @saturnism @googlecloud @kubernetesio 9 Building Containers

  10. 10 @saturnism @googlecloud @kubernetesio Tag your containers! What do these

    containers have in common? helloworld-service:latest debian:9 openjdk:8
  11. 11 @saturnism @googlecloud @kubernetesio Pin Your Versions! RUN apt-get update

    RUN apt-get install curl ← which version??
  12. 12 @saturnism @googlecloud @kubernetesio Combine Run Commands RUN apt-get update

    RUN apt-get install -y --no-install-recommends ... RUN rm -rf /var/lib/apt/lists/* vs. RUN apt-get update && \ apt-get install -y --no-install-recommends ... && \ rm -rf /var/lib/apt/lists/*
  13. 13 @saturnism @googlecloud @kubernetesio Avoid Saving Files RUN curl http://.../apache-tomcat-8.5.20.tar.gz

    | tar xz -C /opt/
  14. 14 @saturnism @googlecloud @kubernetesio Don’t Log to Container Filesystem! Log

    to a volume… docker -v /tmp/log:/log Or, better yet, Send it elsewhere! I prefer STDOUT
  15. 15 @saturnism @googlecloud @kubernetesio One Container, One Process Don't start

    multiple processes in daemon
  16. 16 @saturnism @googlecloud @kubernetesio Don't run as root! It's default…

    :( Specify via USER directive and switch users
  17. 17 @saturnism @googlecloud @kubernetesio What's in that public container? Vulnerabilities

  18. 18 @saturnism @googlecloud @kubernetesio Don't Run as root But I

    think we all do...
  19. 19 @saturnism @googlecloud @kubernetesio 19 Specifically, Java Applications

  20. 20 @saturnism @googlecloud @kubernetesio Use JDK 8u131 or Newer -XX:+UnlockExperimentalVMOptions

  21. 21 @saturnism @googlecloud @kubernetesio Heap vs Native Memory Set -Xmx

    to percentage of memory limit Use a Startup Script
  22. 22 @saturnism @googlecloud @kubernetesio Build Thin Layers Use Multi-Stage Build

    Extract Dependencies to a Layer
  23. 23 @saturnism @googlecloud @kubernetesio Avoid Multi-Module Project When you have

    independent services, Don't put in the same multi-module project...
  24. 24 @saturnism @googlecloud @kubernetesio 24 Running in Kubernetes

  25. 25 @saturnism @googlecloud @kubernetesio Label, label, label! app=helloworld-service version=2.0 serving=true

  26. 26 @saturnism @googlecloud @kubernetesio Let it crash Let Kubernetes restart

    for you
  27. 27 @saturnism @googlecloud @kubernetesio Assume Unreliable Services Don't sequence/orchestrate startups

    Handle failures, or let it crash
  28. 28 @saturnism @googlecloud @kubernetesio L4 vs L7 Know the differences,

    especially for gRPC
  29. 29 @saturnism @googlecloud @kubernetesio kubectl apply --record Record the command

    line, shows in history: kubectl rollout history deployments ...
  30. 30 @saturnism @googlecloud @kubernetesio Readiness Probe, Liveness Probe If feeling

    lazy, always have Readiness Probe!
  31. 31 @saturnism @googlecloud @kubernetesio External Service kind: Service apiVersion: v1

    metadata: name: mydatabase namespace: prod spec: type: ExternalName externalName: my.db.example.com kind: Service apiVersion: v1 metadata: name: mydatabase spec: ports: - protocol: TCP port: 80 targetPort: 12345 Then add your own endpoints!
  32. 32 @saturnism @googlecloud @kubernetesio Graceful Shutdown Lifecycle Hooks Shutdown hooks,

    or listen to SIGTERM
  33. 33 @saturnism @googlecloud @kubernetesio 33 More on Security...

  34. 34 @saturnism @googlecloud @kubernetesio Don't Run as root! apiVersion: v1

    kind: Pod metadata: name: hello-world spec: securityContext: runAsNonRoot: true allowPrivilegeEscalation: false runAsUser: 1000 fsGroup: 2000 containers: # specification of the pod’s containers # ...
  35. 35 @saturnism @googlecloud @kubernetesio Read Only Filesystem apiVersion: v1 kind:

    Pod metadata: name: hello-world spec: securityContext: readOnlyRootFilesystem: true ... containers: ...
  36. 36 @saturnism @googlecloud @kubernetesio Containing Breakouts Containers are are not

    security boundaries! We can try seccomp, apparmor, selinux, but still! annotations: seccomp.security.alpha.kubernetes.io/pod: ... container.apparmor.security.beta.kubernetes.io/hello: ...
  37. 37 @saturnism @googlecloud @kubernetesio PodSecurityPolicy Enforce Security Policy cluster-wide apiVersion:

    extensions/v1beta1 kind: PodSecurityPolicy metadata: name: example spec: privileged: false # Don't allow privileged pods! # The rest fills in some required fields. seLinux: rule: RunAsAny supplementalGroups: rule: RunAsAny runAsUser: rule: 1000 fsGroup: rule: RunAsAny
  38. 38 @saturnism @googlecloud @kubernetesio Service Accounts & RBAC Who can

    access which Kubernetes API to do what?
  39. 39 @saturnism @googlecloud @kubernetesio Use use transient credentials Google Kubernetes

    Engine uses OAuth token
  40. 40 @saturnism @googlecloud @kubernetesio Better yet, tied to user directory

    Google Kubernetes Engine ties to Identity Access Manager
  41. 41 @saturnism @googlecloud @kubernetesio And, don't expose API Server If

    it's on public internet, make sure it's firewalled gcloud container clusters create --enable-master-authorized-networks --master-authorized-networks=...
  42. 42 @saturnism @googlecloud @kubernetesio Network Policy Which pod can establish

    connections to which pod? gcloud container clusters create ... --enable-network-policy
  43. 43 @saturnism @googlecloud @kubernetesio Mutual TLS Stronger than Network Policy

    Use a Service Mesh, like Istio Automatic certificate generation and rotation
  44. 44 @saturnism @googlecloud @kubernetesio 44 CI/CD

  45. 45 @saturnism @googlecloud @kubernetesio Pull Base Layer Because you are

    probably using :latest tag
  46. 46 @saturnism @googlecloud @kubernetesio Build without Cache Because you probably

    install w/o pinning RUN apt-get update && apt-get install ...
  47. 47 @saturnism @googlecloud @kubernetesio Check-in YAMLs! If you didn't remember

    anything else, do this!! If you use a template and don't check-in YAMLs, Must have template + variables values, so you can regenerate it.
  48. 48 @saturnism @googlecloud @kubernetesio 48 Microservices

  49. 49 @saturnism @googlecloud @kubernetesio Observability is not a hindsight! Don't

    wait until something went wrong...
  50. 50 @saturnism @googlecloud @kubernetesio Trace Log Debug

  51. 51 @saturnism @googlecloud @kubernetesio 51 Finally...

  52. 52 @saturnism @googlecloud @kubernetesio Don't manage your own cluster... Use

    a managed service, like Google Kubernetes Engine!
  53. 53 @saturnism @googlecloud @kubernetesio 53 Thanks! http://saturnism.me @saturnism