Docker Cape Town - Extendibility is the secret

Docker Cape Town - Extendibility is the secret

DevOps is the ability to implement flow such as CI/CD, provisioning and automation. There are a lot of possible tools that a team can use to address these problems. Docker is one of the first choices because it is simple and extensible. This won't be a classic talk but more like a chat about some of the applications that use Docker not just via Docker CLI but leveraging the API to make something great. Be ready to share your story and your pain! We are going to get over all of this together!

Fa5fd3405808cc6a9fe4b126b1ec39bd?s=128

Gianluca Arbezzano

March 05, 2019
Tweet

Transcript

  1. 1.

    Gianluca Arbezzano Site Reliability Engineer @InfluxData • https://gianarb.it • @gianarb

    What I like: • I make dirty hacks that look awesome • I grow my vegetables • Travel for fun and work
  2. 5.
  3. 11.

    © 2018 InfluxData. All rights reserved. 11 @gianarb - gianluca@influxdb.com

    apiVersion: extensions/v1beta1 kind: Deployment metadata: name: {{ template "drone.fullname" . }}-agent labels: app: {{ template "drone.name" . }} chart: "{{ .Chart.Name }}-{ .Chart.Version }}" release: "{{ .Release.Name }}" heritage: "{{ .Release.Service }}" component: agent spec: replicas: {{ .Values.agent.replicas }} template: metadata: annotations: checksum/secrets: {{ include (print $.Template.BasePath "/secrets.yaml") . | sha256sum }} {{- if .Values.agent.annotations } {{ toYaml .Values.agent.annotations | indent 8 } {{- end }} labels: app: {{ template "drone.name" . }} release: "{{ .Release.Name }}" component: agent
  4. 18.

    © 2018 InfluxData. All rights reserved. 18 @gianarb - gianluca@influxdb.com

    Docker for Kubernetes Docker for Kubernetes is one of the possible CRI (Container Runtime Interface) implementation, there are a lot of them: ¨ rktlet (Rocket implementation by CoreOS) ¨ containerd-cri (containerd implementation by containerd community) ¨ cri-o (by RedHat) ¨ Probably cloud providers like Amazon, Google they have their own one.
  5. 19.

    © 2018 InfluxData. All rights reserved. 19 @gianarb - gianluca@influxdb.com

    Docker Jenkins Plugin https://wiki.jenkins.io/display/JENKINS/Docker+Plugin
  6. 23.

    © 2018 InfluxData. All rights reserved. 23 @gianarb - gianluca@influxdb.com

    We use docker as replacement for systemd for process management
  7. 24.

    © 2018 InfluxData. All rights reserved. 24 @gianarb - gianluca@influxdb.com

    DIND - Docker in Docker $ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker sh $ docker info Containers: 48 Running: 1 Paused: 0 Stopped: 47 containerd version: 9f2e07b1fc1342d1c48fe4d7bbb94cb6d1bf278b.m runc version: 871ba2e58e24314d1fab4517a80410191ba5ad01 init version: fec3683 Kernel Version: 4.20.13-arch1-1-ARCH Operating System: Arch Linux OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 15.42GiB Name: gianarb
  8. 27.

    © 2018 InfluxData. All rights reserved. 27 @gianarb - gianluca@influxdb.com

    The SDKs ctx := context.Background() cli, err := client.NewClientWithOpts(client.FromEnv) if err != nil { panic(err) } cli.NegotiateAPIVersion(ctx) reader, err := cli.ImagePull(ctx, "docker.io/library/alpine", types.ImagePullOptions{}) if err != nil { panic(err) } io.Copy(os.Stdout, reader)
  9. 28.

    © 2018 InfluxData. All rights reserved. 28 @gianarb - gianluca@influxdb.com

    Docker swarm and SwarmKit https://github.com/docker/swarmkit SwarmKit is a toolkit for orchestrating distributed systems at any scale. It includes primitives for node discovery, raft-based consensus, task scheduling and more. $ docker swarm init
  10. 29.

    © 2018 InfluxData. All rights reserved. 29 @gianarb - gianluca@influxdb.com

    New standards around containers ¨ CRI (Container Runtime Interface) ¨ CNI (container networking interface) ¨ CSI (container storage interface) ¨ OCI ¨ Runtime Specification ¨ Image Specification
  11. 30.

    © 2018 InfluxData. All rights reserved. 30 @gianarb - gianluca@influxdb.com

    OCI/Image Specification ¨ docker/buildkit ¨ containers/skopeo
  12. 31.

    © 2018 InfluxData. All rights reserved. 31 @gianarb - gianluca@influxdb.com

    CSI/Kubernetes Persistent Volume ¨ kubernetes-sigs/aws-ebs-csi-driver ¨ kubernetes-sigs/gcp-filestore-csi-driver ¨ aws/csi-driver-amazon-efs ¨ Azure/kubernetes-volume-drivers ¨ … a lot more, almost every cloud provider or services they provide
  13. 32.

    © 2018 InfluxData. All rights reserved. 32 @gianarb - gianluca@influxdb.com

    TestContainers github.com/testcontainers is on organization that groups different libraries, cross languages, to programmatically manage containers inside your tests. The most famous one is the java library testcontainers/testcontainers-jav a public class RedisBackedCacheIntTest { private RedisBackedCache underTest; @Rule public GenericContainer redis = new GenericContainer<>( "redis:5.0.3-alpine" ).withExposedPorts (6379); @Before public void setUp() { String address = redis. getContainerIpAddress (); Integer port = redis. getFirstMappedPort (); underTest = new RedisBackedCache(address, port); } @Test public void testSimplePutAndGet () { underTest. put("test", "example"); String retrieved = underTest. get("test"); assertEquals( "example", retrieved); } }
  14. 33.

    © 2018 InfluxData. All rights reserved. 33 @gianarb - gianluca@influxdb.com

    TestContainers I maintain the Golang version of the library testcontainers/testcontainer s-go package main import ( "context" "fmt" "net/http" "testing" testcontainers "github.com/testcontainers/testcontainers-go" ) func TestNginxLatestReturn(t *testing.T) { ctx := context.Background() req := testcontainers.ContainerRequest{ Image: "nginx", ExposedPorts: []string{"80/tcp"}, } nginxC, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{ ContainerRequest: req, Started: true, }) defer nginxC.Terminate(ctx) ip, err := nginxC.Host(ctx) port, err := nginxC.MappedPort(ctx, "80") resp, err := http.Get(fmt.Sprintf("http://%s:%s", ip, port.Port())) if resp.StatusCode != http.StatusOK { t.Errorf("Expected status code %d. Got %d.", http.StatusOK, resp.StatusCode) } }
  15. 34.

    © 2018 InfluxData. All rights reserved. 34 @gianarb - gianluca@influxdb.com

    gianarb/orbiter curl -v -X POST \ http://localhost:8000/v1/orbiter/handle/infra_scale/docker/up Orbiter is an easy to run autoscaler for Docker Swarm. It is designed to work out of the box.
  16. 35.

    © 2018 InfluxData. All rights reserved. 35 @gianarb - gianluca@influxdb.com

    Reach out: @gianarb gianluca@influxdb.com https://gianarb.it Any question?