is so complicated • Only Google & Netflix have the scale to need it • You need a dedicated team to manage it • I’m happy with [chef/ansible/terraform] ♂
of software workloads ♻ • Lifecycle • Stateful, Stateless, Batch • Provision of storage • Distribution across computing nodes • Networking & access from the outside world • Security, Policies & Access Control • Declaration of how to run workloads in the cloud
the desired state of the system. ◦ Analogous to an instance of a resource in the system ◦ They must have a Kind, Group and Version ◦ They are persisted in etcd. apiVersion: networking.k8s.io/v1beta1 kind: Ingress Group Version
docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 8c315a54ed41 albertoimpl/myapp "java -jar /opt/myap…" 12 seconds ago Up 11 seconds 0.0.0.0:8080->8080/tcp cranky_khayyam
{ begin: name := fmt.Sprintf("%s_%s", left[rand. Intn(len(left))], right[rand. Intn(len(right))]) if name == "boring_wozniak" /* Steve Wozniak is not boring */ { goto begin } if retry > 0 { name = fmt.Sprintf("%s%d", name, rand.Intn(10)) } return name } https://github.com/moby/moby/blob/master/pkg/namesgenerator/names-generator.go#L844
REPOSITORY TAG IMAGE ID CREATED SIZE albertoimpl/myapp latest 6d69e5b15226 4 seconds ago 142MB openjdk 8 e8d00769c8a8 6 days ago 488MB gcr.io/distroless/java 8 2ee039e7a421 49 years ago 125MB
REPOSITORY TAG IMAGE ID CREATED SIZE albertoimpl/myapp latest 8314b5c99ec7 4 minutes ago 107MB openjdk 8 e8d00769c8a8 6 days ago 488MB adoptopenjdk/openjdk8 alpine-slim a3562aa0b991 4 months ago 90.2MB
images REPOSITORY TAG IMAGE ID CREATED SIZE gcr.io/distroless/java 8 2ee039e7a421 49 years ago 125MB myapp-jib 0.0.1-SNAPSHOT 272b59084a4c 49 years ago 142MB
original and most used registry, free for public, paid for private. • GRC/ACR/ECR Are the biggest cloud providers registries, if you are using their hosted kubernetes container services, you should use their registries. • Harbor Open Source registry part of the CNCF. On premise registry. Vulnerability Scanner.
Use a managed service from your cloud provider of choice • Otherwise use an on-prem distribution (OpenShift, PKS, Rancher etc) • For the brave and curious - kubeadm • If you want to really understand how it fits together - Kubernetes the hard way by Kelsey Hightower
cluster-info Kubernetes master is running at https://127.0.0.1:60694 KubeDNS is running at https://127.0.0.1:60694/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
STATUS AGE default Active 56s kube-node-lease Active 60s kube-public Active 60s kube-system Active 60s Namespaces provide a scope for names and it's a way to divide cluster resources.
pod/myapp created % k get pods NAME READY STATUS RESTARTS AGE myapp 0/1 ImagePullBackOff 0 1m % k get pods NAME READY STATUS RESTARTS AGE myapp 1/1 Running 0 1m
Pods and a policy to access them. The main types are: • ClusterIP: Exposes only inside the cluster. • NodePort: Exposes a port through the node to the world. • LoadBalancer: Exposes the Service externally using a cloud provider’s load balancer. Service
service/service-myapp-jib created % k get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 92m service-myapp-jib NodePort 10.111.36.24 <none> 8080:31106/TCP 6s % k port-forward service/service-myapp-jib 8080:8080 Forwarding from 127.0.0.1:8080 -> 8080 Forwarding from [::1]:8080 -> 8080 % curl localhost:8080 Hello, All
replicaset.apps/myapp created % k get po,rs NAME READY STATUS RESTARTS AGE pod/myapp 1/1 Running 0 101m pod/myapp-lbb82 1/1 Running 0 43s pod/myapp-m82tv 1/1 Running 0 43s NAME DESIRED CURRENT READY AGE replicaset.extensions/myapp 3 3 3 43s
"myapp" deleted % k get po,svc,rs NAME READY STATUS RESTARTS AGE pod/myapp-l5tfq 1/1 Running 0 18s pod/myapp-lbb82 1/1 Running 0 4m35s pod/myapp-m82tv 1/1 Running 0 4m35s NAME DESIRED CURRENT READY AGE replicaset.extensions/myapp 3 3 3 4m35s
READY STATUS RESTARTS AGE pod/myapp-l5tfq 1/1 Running 0 4m11s pod/myapp-lbb82 1/1 Running 0 8m28s pod/myapp-m82tv 1/1 Running 0 8m28s NAME DESIRED CURRENT READY AGE replicaset.extensions/myapp 3 3 3 8m28s NAME READY UP-TO-DATE AVAILABLE AGE deployment.extensions/myapp 3/3 3 3 2s
✓ Okteto manifest (okteto.yml) created % okteto up Deployment app-okteto doesn't exist in namespace default. Do you want to create a new one? [y/n]: y ✓ Development environment activated ✓ Files synchronized Namespace: default Name: app-okteto Forward: 8080 -> 8080 8088 -> 8088
Robust and secure - uses cloud load balancer service ⚠ Will incur more costs from cloud providers ⚠ Vendor specific configuration ⚠ Less configurable than Ingress ⚠ LoadBalancer not available in on-prem k8s (needs metal-lb, a 3rd party LoadBalancer implementation)
Registry Applicatio n Container ReplicaSet Deployment Service Ingress Pod Application Container Image Registry Application Container ReplicaSet Deployment Service Ingress
ingress.networking.k8s.io/provider-ingress created % k get svc,ingress NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d1h service/service-myapp-provider NodePort 10.107.90.53 <none> 8080:31230/TCP 2d21h NAME HOSTS ADDRESS PORTS AGE ingress.extensions/provider-ingress provider.test.app.com 107.178.254.228 80 75s % curl 107.178.254.228/customers ["Laura","Bella","Olga"]
sh # cat config/DB_PASSWORD supersecret • Encrypting the data on rest: https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/ • Using Sealed Secrets that allow us to encrypt everything on git: https://github.com/bitnami-labs/sealed-secrets • Using Vault to store them: https://github.com/coreos/vault-operator
really help us here • Ingress can provide course-grained auth for a service • Implement an OAuth2 or Openid Connect flow ◦ On Prem: Hydra, Dex, UAA ◦ Cloud: Google, Github etc • Istio can secure communication between pods using mTLS & policy rules
humans • ServiceAccounts identify a process • Roles describe an action against a resource eg create Pod • RoleBinding describes a set of Users or ServiceAccounts that a Role applies to • Namespaces are a logical grouping of resources User: Jon Can create & read Resource: Pod Namespace: prod Resource: Pod / BookingApp ServiceAccount: DevPods Can read Resource: Secret / DevSecret
dons/fluentd-elasticsearch % k apply -f . service/elasticsearch-logging created serviceaccount/elasticsearch-logging created clusterrole.rbac.authorization.k8s.io/elasticsearch-logging created clusterrolebinding.rbac.authorization.k8s.io/elasticsearch-logging created statefulset.apps/elasticsearch-logging created configmap/fluentd-es-config-v0.2.0 created serviceaccount/fluentd-es created clusterrole.rbac.authorization.k8s.io/fluentd-es created clusterrolebinding.rbac.authorization.k8s.io/fluentd-es created daemonset.apps/fluentd-es-v2.7.0 created deployment.apps/kibana-logging created service/kibana-logging created
management: endpoints: web: exposure: include: "prometheus,info,health" % curl localhost:8001/actuator/prometheus # HELP jvm_memory_committed_bytes The amount of memory in bytes that is committed for the Java virtual machine to use # TYPE jvm_memory_committed_bytes gauge jvm_memory_committed_bytes{area="heap",id="PS Survivor Space",} 1.2582912E7
1. See https://thenewstack.io/kubernetes-design-and-development-explained/ Describe Desired State Operator Control Plane Persist Desired State Controller Watch for changes of interest Ensure cluster is in desired state
albertoimpl
h455h1
ryukobayashi
teamlab
nekowen
mraible
chiroruxx
faithandbrave
sonatard
polamjag
kishida
pospome
bkuhlmann
tanoku
addyosmani
samlambert
frogandcode
robhawkes
shpigford
gr2m
lara
mclloyd
mthomps
garrettdimon
bkeepers
