Slide 1

Slide 1 text

Google Cloud Platform logo Kubernetes TLS/Let’s Encrypt Seattle Kubernetes Meetup 2017-06-01 Ahmet Alp Balkan Software Engineer, Developer Programs @ Google @ahmetb 1

Slide 2

Slide 2 text

Google Cloud Platform What is TLS Transport-layer* security * OSI Layer 4 (TCP/IP) SSL v2.0 in Netscape Navigator (1995) SSL v3.0 redesign of the protocol (1996) TLS v1.0 IETF standard published (1999) TLS v1.3 still draft as of 2016. Required by HTTP/2 in most implementations. Photo credit: 24hourwebhostingsupport.com 2

Slide 3

Slide 3 text

Google Cloud Platform What is HTTPS TLS TLS TLS TLS TLS HTTP HTTP Photo credit: windchillfactory.com 3

Slide 4

Slide 4 text

Google Cloud Platform Public-key cryptography 101 4

Slide 5

Slide 5 text

Google Cloud Platform Public-key cryptography 101 Alice Bob ALICE_PRIV ALICE_PUB Photo credit: indelwebastomarine.com 5

Slide 6

Slide 6 text

Google Cloud Platform Public-key cryptography 101 Alice Bob Photo credit: indelwebastomarine.com 6

Slide 7

Slide 7 text

Google Cloud Platform Public-key cryptography 101 Alice Bob secret text Photo credit: indelwebastomarine.com 7

Slide 8

Slide 8 text

Google Cloud Platform Public-key cryptography 101 Alice Bob Photo credit: indelwebastomarine.com 8

Slide 9

Slide 9 text

Google Cloud Platform Public-key cryptography 101 Alice Bob Photo credit: indelwebastomarine.com secret text 9

Slide 10

Slide 10 text

Google Cloud Platform Why TLS? Photo credit: Laura Poitras/Praxis Film y u no TLS? 10

Slide 11

Slide 11 text

Google Cloud Platform Why TLS? Users want Privacy Developers want Privacy of users, integrity of the transmitted data Services need Identity/Authentication (with mutual TLS)

Slide 12

Slide 12 text

Google Cloud Platform Where TLS? tl;dr: everywhere service HTTPs service HTTPs HTTPs users apps service HTTPs Load Balancer HTTPs gRPC+TLS 12

Slide 13

Slide 13 text

Google Cloud Platform Challenges with TLS for external traffic • Find a Certificate Authority (CA) • Prepare a Certificate Signing Request (learn how to create a CSR) • Manual verification process (fax, …) • Pay for the certificate • Download/store/install certs safely • Make sure the certs do not expire • Hope CA doesn’t get compromised • Hope you don’t need to revoke your cert 13

Slide 14

Slide 14 text

Google Cloud Platform ...Challenges with TLS for internal traffic • Host a Certificate Authority (CA) and PKI • Create a trust chain • Create certs for each service • develop in-house software to automate • Make sure certs do not expire • Create and store keys safely, set up the server • Hope you don’t get compromised 14

Slide 15

Slide 15 text

Google Cloud Platform Internal TLS 15

Slide 16

Slide 16 text

Google Cloud Platform Kubernetes TLS Certificate Authority Every Kubernetes cluster has a root CA. • Used by cluster components to validate trust • You can have cluster root CA sign your certificates too. 16

Slide 17

Slide 17 text

Google Cloud Platform Requesting a certificate from Kubernetes root CA 1. Create a CertificateSigningRequest 2. Approve the request (manual) 3. Server: download the cert from k8s API & use 4. Client: trust the k8s cluster root CA • k8s CA bundle is available on every Pod: /var/run/secrets/kubernetes.io/serviceaccount/ca.tls More info: https://kubernetes.io/docs/tasks/tls/ managing-tls-in-a-cluster/ 17

Slide 18

Slide 18 text

Google Cloud Platform Current issues with Kubernetes cluster root CA • Write code to automate process • check out kelseyhightower/certificate-init-container • Write code to • refresh the certificates • signal pods to reload the cert or restart the pods • handle cluster root CA certificate rotation • CA used for cluster components is the same as user workloads. • Suboptimal developer experience for now. ¯\_(ツ)_/¯ 18

Slide 19

Slide 19 text

Google Cloud Platform A better alternative A service mesh can run its own CA/PKI, and encrypt all communication between microservices automatically: • Istio: http://istio.io • Linkerd: http://linkerd.io app sidecar app sidecar localhost localhost POD A POD B TLS 19

Slide 20

Slide 20 text

Google Cloud Platform External TLS 20

Slide 21

Slide 21 text

Google Cloud Platform Let’s Encrypt Started in 2014, launched in 2016. Problem: • Getting a TLS certificate is too messy and expensive. • Only ~50% of the web traffic is encrypted today (2017) Solution: • Create a free CA that is fully automated. 21

Slide 22

Slide 22 text

Google Cloud Platform Let’s Encrypt – How it works Uses open IETF ACME protocol for issuing certificates. ACME client ACME server request certificate challenges complete challenge verify challenge issue 90-day certificate (or denial) 22

Slide 23

Slide 23 text

Google Cloud Platform ACME – Challenges • HTTP: place a file on your server (easy) e.g. example.com/.well-known/j4kw5n23 • DNS: add TXT record to your domain (easy) TXT _acme-challenge.example.com j4kw5n23 • TLS-SNI: serve a self-signed certificate with the challenge token with two subjectAltNames (difficult) 23

Slide 24

Slide 24 text

Google Cloud Platform Your apps can request Let’s Encrypt certificates. Challenges: • How to survive pod restarts • Coordinating between multiple replicas Example: • Caddy is a HTTP/2 server with automatic TLS (written in Go). It uses the github.com/xenolf/lego ACME implementation. (caddyserver.com) • Many languages have ACME libraries. You can build ACME in your application 24

Slide 25

Slide 25 text

Google Cloud Platform State of Let’s Encrypt in Kubernetes Fragmented. Popular Fork of below Merged solution, to be incubated 25

Slide 26

Slide 26 text

Google Cloud Platform Before you use these • You probably should not use either of these yet. • until the merged cert-manager moves to kubernetes-incubator and has stable releases. • Suboptimal user experience. :( • Make sure you have monitoring in place for expirations. Set alerting for certs close to expiration and refresh certificates periodically. 26

Slide 27

Slide 27 text

Google Cloud Platform Most practical option today Actually go buy a TLS certificate from one of the CAs to terminate external TLS. Use service mesh (linkerd, istio etc) to encrypt your internal traffic. 27

Slide 28

Slide 28 text

Google Cloud Platform Try it out – kube-lego on Google Container Engine https://github.com/jetstack/kube-lego/tree/master/examples/gce Took me 20 minutes to get a HTTPs website running and I had to troubleshoot constantly (and I used it before). 28

Slide 29

Slide 29 text

Google Cloud Platform Thank you @ahmetb 29

Slide 30

Slide 30 text

Google Cloud Platform Appendix/Backup slides 30

Slide 31

Slide 31 text

Google Cloud Platform Step 1: Create CSR server-csr.json: { "hosts": [ "web.default.svc.cluster.local" ], "CN": "web.default.svc.cluster.local", "key": { "algo": "ecdsa", "size": 256 } } Turns this into a server.csr and server-key.pem: $ cfssl genkey server-csr.json | cfssljson -bare server 31

Slide 32

Slide 32 text

Google Cloud Platform $ cat <

Slide 33

Slide 33 text

Google Cloud Platform $ kubectl certificate approve web.default $ kubectl get csr NAME AGE REQUESTOR CONDITION web.default 45s [email protected] Approved,Issued Step 3: Manually approve CSR 33

Slide 34

Slide 34 text

Google Cloud Platform Secret is stored on status.certificate field: $ kubectl get csr web.default -o jsonpath='{.status.certificate}' \ | base64 -d > server.crt Create a TLS secret with the key/cert pair: $ kubectl create secret tls web-tls --cert server.crt --key server-key.pem Then mount the web-tls secret as volume and use it. Step 4: Extract the certificate 34