Slide 1

Slide 1 text

Gianluca Arbezzano Site Reliability Engineer @InfluxData ● https://gianarb.it ● @gianarb What I like: ● I make dirty hacks that look awesome ● I grow my vegetables ● Travel for fun and work

Slide 2

Slide 2 text

@gianarb - [email protected]

Slide 3

Slide 3 text

@gianarb - [email protected]

Slide 4

Slide 4 text

@gianarb - [email protected]

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

© 2018 InfluxData. All rights reserved. 6 @gianarb - [email protected]

Slide 7

Slide 7 text

Why Kubernetes is so powerful, complex and widely adopted? widely

Slide 8

Slide 8 text

Why AWS is so expensive? widely

Slide 9

Slide 9 text

What do you do to justify these costs?

Slide 10

Slide 10 text

© 2018 InfluxData. All rights reserved. 10 @gianarb - [email protected]

Slide 11

Slide 11 text

© 2018 InfluxData. All rights reserved. 11 @gianarb - [email protected] apiVersion: extensions/v1beta1 kind: Deployment metadata: name: {{ template "drone.fullname" . }}-agent labels: app: {{ template "drone.name" . }} chart: "{{ .Chart.Name }}-{ .Chart.Version }}" release: "{{ .Release.Name }}" heritage: "{{ .Release.Service }}" component: agent spec: replicas: {{ .Values.agent.replicas }} template: metadata: annotations: checksum/secrets: {{ include (print $.Template.BasePath "/secrets.yaml") . | sha256sum }} {{- if .Values.agent.annotations } {{ toYaml .Values.agent.annotations | indent 8 } {{- end }} labels: app: {{ template "drone.name" . }} release: "{{ .Release.Name }}" component: agent

Slide 12

Slide 12 text

API are the keys for your success! Image credit: Pixabay

Slide 13

Slide 13 text

© 2018 InfluxData. All rights reserved. 13 @gianarb - [email protected]

Slide 14

Slide 14 text

© 2018 InfluxData. All rights reserved. 14 @gianarb - [email protected]

Slide 15

Slide 15 text

© 2018 InfluxData. All rights reserved. 15 @gianarb - [email protected] containerd.io

Slide 16

Slide 16 text

© 2018 InfluxData. All rights reserved. 16 @gianarb - [email protected]

Slide 17

Slide 17 text

© 2018 InfluxData. All rights reserved. 17 @gianarb - [email protected]

Slide 18

Slide 18 text

© 2018 InfluxData. All rights reserved. 18 @gianarb - [email protected] Docker for Kubernetes Docker for Kubernetes is one of the possible CRI (Container Runtime Interface) implementation, there are a lot of them: ¨ rktlet (Rocket implementation by CoreOS) ¨ containerd-cri (containerd implementation by containerd community) ¨ cri-o (by RedHat) ¨ Probably cloud providers like Amazon, Google they have their own one.

Slide 19

Slide 19 text

© 2018 InfluxData. All rights reserved. 19 @gianarb - [email protected] Docker Jenkins Plugin https://wiki.jenkins.io/display/JENKINS/Docker+Plugin

Slide 20

Slide 20 text

© 2018 InfluxData. All rights reserved. 20 @gianarb - [email protected]

Slide 21

Slide 21 text

© 2018 InfluxData. All rights reserved. 21 @gianarb - [email protected]

Slide 22

Slide 22 text

© 2018 InfluxData. All rights reserved. 22 @gianarb - [email protected]

Slide 23

Slide 23 text

© 2018 InfluxData. All rights reserved. 23 @gianarb - [email protected] We use docker as replacement for systemd for process management

Slide 24

Slide 24 text

© 2018 InfluxData. All rights reserved. 24 @gianarb - [email protected] DIND - Docker in Docker $ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker sh $ docker info Containers: 48 Running: 1 Paused: 0 Stopped: 47 containerd version: 9f2e07b1fc1342d1c48fe4d7bbb94cb6d1bf278b.m runc version: 871ba2e58e24314d1fab4517a80410191ba5ad01 init version: fec3683 Kernel Version: 4.20.13-arch1-1-ARCH Operating System: Arch Linux OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 15.42GiB Name: gianarb

Slide 25

Slide 25 text

docker run \ -v /var/run/docker.sock:/var/run/docker.sock

Slide 26

Slide 26 text

dockerd -H tcp://10.120.0.12

Slide 27

Slide 27 text

© 2018 InfluxData. All rights reserved. 27 @gianarb - [email protected] The SDKs ctx := context.Background() cli, err := client.NewClientWithOpts(client.FromEnv) if err != nil { panic(err) } cli.NegotiateAPIVersion(ctx) reader, err := cli.ImagePull(ctx, "docker.io/library/alpine", types.ImagePullOptions{}) if err != nil { panic(err) } io.Copy(os.Stdout, reader)

Slide 28

Slide 28 text

© 2018 InfluxData. All rights reserved. 28 @gianarb - [email protected] Docker swarm and SwarmKit https://github.com/docker/swarmkit SwarmKit is a toolkit for orchestrating distributed systems at any scale. It includes primitives for node discovery, raft-based consensus, task scheduling and more. $ docker swarm init

Slide 29

Slide 29 text

© 2018 InfluxData. All rights reserved. 29 @gianarb - [email protected] New standards around containers ¨ CRI (Container Runtime Interface) ¨ CNI (container networking interface) ¨ CSI (container storage interface) ¨ OCI ¨ Runtime Specification ¨ Image Specification

Slide 30

Slide 30 text

© 2018 InfluxData. All rights reserved. 30 @gianarb - [email protected] OCI/Image Specification ¨ docker/buildkit ¨ containers/skopeo

Slide 31

Slide 31 text

© 2018 InfluxData. All rights reserved. 31 @gianarb - [email protected] CSI/Kubernetes Persistent Volume ¨ kubernetes-sigs/aws-ebs-csi-driver ¨ kubernetes-sigs/gcp-filestore-csi-driver ¨ aws/csi-driver-amazon-efs ¨ Azure/kubernetes-volume-drivers ¨ … a lot more, almost every cloud provider or services they provide

Slide 32

Slide 32 text

© 2018 InfluxData. All rights reserved. 32 @gianarb - [email protected] TestContainers github.com/testcontainers is on organization that groups different libraries, cross languages, to programmatically manage containers inside your tests. The most famous one is the java library testcontainers/testcontainers-jav a public class RedisBackedCacheIntTest { private RedisBackedCache underTest; @Rule public GenericContainer redis = new GenericContainer<>( "redis:5.0.3-alpine" ).withExposedPorts (6379); @Before public void setUp() { String address = redis. getContainerIpAddress (); Integer port = redis. getFirstMappedPort (); underTest = new RedisBackedCache(address, port); } @Test public void testSimplePutAndGet () { underTest. put("test", "example"); String retrieved = underTest. get("test"); assertEquals( "example", retrieved); } }

Slide 33

Slide 33 text

© 2018 InfluxData. All rights reserved. 33 @gianarb - [email protected] TestContainers I maintain the Golang version of the library testcontainers/testcontainer s-go package main import ( "context" "fmt" "net/http" "testing" testcontainers "github.com/testcontainers/testcontainers-go" ) func TestNginxLatestReturn(t *testing.T) { ctx := context.Background() req := testcontainers.ContainerRequest{ Image: "nginx", ExposedPorts: []string{"80/tcp"}, } nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ ContainerRequest: req, Started: true, }) defer nginxC.Terminate(ctx) ip, err := nginxC.Host(ctx) port, err := nginxC.MappedPort(ctx, "80") resp, err := http.Get(fmt.Sprintf("http://%s:%s", ip, port.Port())) if resp.StatusCode != http.StatusOK { t.Errorf("Expected status code %d. Got %d.", http.StatusOK, resp.StatusCode) } }

Slide 34

Slide 34 text

© 2018 InfluxData. All rights reserved. 34 @gianarb - [email protected] gianarb/orbiter curl -v -X POST \ http://localhost:8000/v1/orbiter/handle/infra_scale/docker/up Orbiter is an easy to run autoscaler for Docker Swarm. It is designed to work out of the box.

Slide 35

Slide 35 text

© 2018 InfluxData. All rights reserved. 35 @gianarb - [email protected] Reach out: @gianarb [email protected] https://gianarb.it Any question?